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@ Handwritten keyboardless-entry computer system. 

© A keyboardless-entry computer system includes a transparent input screen (18) that generates positional 
information when contacted by a stylus (16) and a display screen (20) mounted below the input screen so that a 
displayed symbol can be viewed through the input screen. Handwritten symbols can be entered using the stylus 
and the corresponding positional information is input to a processor to identify the symbol. When a predeter- 
mined command symbol is identified, a window area is displayed on the display screen in which a handwritten 
symbol or symbols can be entered. When the entry in the window is complete, a further command closes the 
window and, for example, inserts the contents of the window into a document displayed on the display screen at 
a position determined by the location of the window opening command in the document on the screen. 
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The present invention relates generally to a keyboardless input system to a computer, and when 
combined with a central processing unit, to a keyboardless-entry computer system. More particularly, the 
present invention relates to an information storage, manipulation and transfer device on which text, data, 
computer commands and functions can be entered by writing alphanumeric or any other characters and 
5 symbols by hand with a penlike stylus on an Input/Output (I/O) screen. In a preferred embodiment the I/O 
screen includes a transparent touch screen incorporated over a substantially flat output display. The present 
invention in its preferred embodiment is a self-contained computer system but can also function as a 
peripheral to a host computer. 

Large amounts of information and sophisticated applications software are now available on conventional 
10 keyboard computers. The utility of this information and of application software could be greatly increased if 
text and data could be entered and applications software manipulated by writing in a normal fashion directly 
on a flat display. Thus, there is a need to allow the utility of computer technology to be extended for use by 
non-keyboard oriented individuals. There is also a need for a portable computer system that is lightweight, 
reliable, accurate, inexpensive and permits use while standing or walking. One way to reduce expense and 
75 size and increase utility is to employ a keyboardless entry system, such as a touch screen. However, this 
type of input device does not easily allow accurate detailed input within a real time framework with high 
resolution in a manner which is familiar and natural to the user. 

Many positioning technologies can be used to meet the requirements of the position sensing input 
technology. Essentially these requirements include accuracy, resolution and speed. The technologies 
20 include: mechanical, electrostatic, electromagnetic, acoustic, optical, and inertial. The desire in this system 
is to have its use as similar as possible to writing with pen or pencil on paper. One problem is proximity - a 
pen on paper only leaves a trail when actually in contact. Many of these technologies require an additional 
"pen down" sensor which is awkward to use in many commercial pens. Another problem is writing angle -- 
a pen leaves the same trail independent of writing angle. Many of these technologies have the position 
25 detector displaced from the pen tip, so pen angle causes erroneous displacements. Beyond these general 
problems, each technology has numerous advantages and disadvantages in (1) the pen: size, weight, shape 
and whether it needs to be powered and/or wired, and (2) the writing surface: transparency, smoothness, 
"feel", and whether or not physical contact is needed (as opposed to pressure transmitted through 
overlaying sheets of paper). 

30 A number of self-contained devices for viewing and processing large amounts of information are known. 
Most employ optical, magnetic or solid-state electronic storage means to store data. Illustrative of this body 
of art is U.S. Patent 4,159,417 to Rubincam which discloses a portable electronic book configured to 
provide selective page by page call-up of large amounts of digital data and displays it on a flat, solid-state 
screen. The preferred embodiment in the Rubincam patent uses an insertable holographic card, which may 

35 contain several hundred pages of text in digital form, as the main storage means. Rubincam's device, 
however, does not allow new information or text to be entered or manipulated. 

In U.S. Patent 4,016,542 to Azure an electronic data collection system is disclosed which employs a 
solid state Random Access Memory (RAM) for its primary memory. This patent, which discloses a 
conventional keyboard for data entry, is directed to a hand-held portable data storage and transmission 

40 system, as well as an LED display and various Input/Output (I/O) connectors. 

U.S. Patent 3,487,731 to Frank discloses a means of converting handwriting into character data through 
the use of a computer system. The disclosed invention is based on matrix pattern matching and does not 
employ any coincident display technology. 

U.S. Patent No. 4,491,960 to Brown shows a handwritten symbol recognition system in which an array 

45 of image Points, in the form of a raster line sampling, is converted into segment-oriented lists which are 
filtered and compressed to obtain topologic features which are then analyzed with a logic tree decision 
mechanism. 

U.S. Patent No. 4,262,281 to Buckle et al. discloses a handwriting recognition device. The disclosed 
embodiment is for use with a host computer and does not employ coincident display technology. 
50 U.S. Patent No. 4,475,239 to Van Raamsdonk discloses a text editing apparatus. The '239 patent calls 
for the use of paper as a medium for the entering of editing functions and requires a conventional keyboard 
for the inputting of text. 

U.S. Patent 4,521,909 to Wang shows a dual level pattern recognition system. The system is designed 
for use with an optical instrument. 
55 U.S. Patent 4,520,357 to Castleberry et al. discloses an electroscopic information display and entry 
system with writing stylus. The disclosed embodiment does not claim to have the speed or accuracy to 
enable handwritten character recognition. 
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Additional prior art which discloses portable electronic devices that provide large amounts of various 
types of stored information include U.S. Patents 4,218,760 to Levy; 4,4115,486 to Laine; and 3,932,859 to 
Kriakides et al. The Levy and Kriakides et al. patents pertain to electronic dictionaries, while the Laine 
patent discloses a programmable television reminder system. None of these devices disclose the use of a 
5 handwritten input. 

In U.S. Patents 4,071,691, 4,129,747, 4,198,539, 4,293,734, 4,302,011, 4,353,552, 4,371,746 and 
4,430,917 to William Pepper, Jr. various methods or machine-human interfaces using finger touch are 
disclosed. The preferred embodiments in each of these inventions lack sufficient speed and resolution to 
allow handwritten character recognition with a stylus and are designed for other purposes. U.S. Patent 

w 4,318,096 to Pepper teaches the use of a conductive stylus. The '096 patent pertains to graphic design and 
allows line width and line intensity to vary by applying pressure on the stylus with the results displayed on a 
conventional CRT screen. U.S. Patent 3,699,439 to Turner and U.S. Patent 4,055,726 to Turner et al 
discloses two methods for electronic position sensing through the use of a probe. 

The invention in its various aspects is defined in the appended claims to which reference should now 

is be made. 

A preferred embodiment of the invention which is described in more detail below with reference to the 
drawings comprises a unique keyboardless computer system which has the ability to recognise and display 
Handwritten Symbols and cause the computer to display Font Symbols and, if desired, to execute editing 
functions pursuant to Editing Symbols, quickly, easily and at reasonable cost. 

20 This preferred embodiment includes a computer housing with a flat display panel on which a user may 
"write" with a stylus, and has a capability to recognise Handwritten Symbols written on the panel with the 
Stylus and convert them to displayed Font Symbols, and/or to execute Editing Functions with Editing 
Symbols, all with a minimum of technical complexity for the user. Once the keyboardless, portable 
computer is loaded with the desired information and applications software, information and software can be 

25 used and responded to without requiring skills or knowledge related to state-of-the-art computers or other 
data source. The ease-of-use of the input technology of the apparatus enhances the utility of the computer 
for keyboard oriented individuals. Its portability also allows it to be used in applications and settings in 
which portable keyboard computers are awkward, difficult or impossible to use. For example, a multiplicity 
of blank, fully or partly completed forms may be stored in the portable computer memory. In a hospital, 

30 "sheets" of patient data can be stored in the memory of the portable computer, called up by a nurse as the 
nurse makes rounds and relevant data, such as blood pressure, temperature, etc., can then be entered 
manually with a stylus. These corrected or expanded forms can then be down-loaded into a central 
computer memory. 

The requirements of the position sensing input technology are accuracy (Point to Point), resolution 

35 (absolute position) and speed (Points per unit time) to adequately define the written Stroke for recognition 
analysis. For the recognition apparatus and methods presently used, as described below, the present 
minimum requirements are: accuracy of 0.125mm (0.005 inch), resolution of 0.375mm (0.015 inch) and 
speed of 150 Points per second. This accuracy allows a 6mm (1/4") high writing line with over 10 raw input 
Points along a Stroke of a small letter. The resolution provides positioning of the symbol to within two pixels 

40 on a present display of 640 pixels to 225mm (9 inches). The speed permits about 50 raw input Points for a 
rapidly written single letter (1/3 second). 

One embodiment of the present invention comprises a transparent input screen. As the user writes 
alphanumeric or other characters or symbols on the input screen, the character is represented as a stream 
of Points emulating written input with pen on paper. Once the discrete alphanumeric and other characters or 

45 symbols are complete, they are translated into computer text or computer commands that can be displayed 
on a display screen at a location that is preferably beneath the area on the input screen where they were 
entered. The embodiment also comprises a pattern recognition algorithm which allows the translation of any 
written character or symbol, such as ideographs and scientific symbols, into computer text. 

In a particular, presently preferred embodiment, a keyboardless computer is configured as a manipula- 

50 tion and display device comprising a transparent touch screen and associated electronics placed over an 80 
column by 25 line or larger display screen; a stylus for entry of data; a microprocessor and storage means; 
artificial intelligence/pattern recognition software and editing software; and a battery power system; and 
other I/O means. 

As used herein, "Handwritten Symbols" are any symbols capable of being handwritten and having 
55 communicative effect. By way of example, and not limitation, numbers, letters, "Kanji" (Japanese ideo- 
grams) or other language symbols, editing symbols and engineering, scientific, architectural and mathemat- 
ical symbols are Handwritten Symbols. Other examples of Handwritten Symbols are free-hand drawings or 
signatures or any other such written information uniquely configured by a particular writer. Handwritten 
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Symbols may also include Editing Symbols (defined below). 

As used herein, "Font Symbols" are computer-generated symbols which are displayed in a predeter- 
mined font format. By way of example and not limitation, alphanumeric symbols may be Font Symbols and 
displayed-in numerous font formats. Japanese or Chinese "ideograms" may also be Font Symbols, as may 
5 be engineering, scientific, mathematical, architectural or other such characters. Other examples of Font 
Symbols include any form which can be stored and displayed by a computer, e.g., a drawing of a car or a 
house. 

As used herein, an "Editing Symbol" is a symbol (such as a caret, horizontal line, short vertical line, 
long vertical line, etc.) which is intended, when recognized, to cause the computer to execute a particular 

10 Editing Function (defined below), such as insert test (caret), delete text (horizontal line) delete a letter (short 
vertical line) or move a margin (long vertical line), to list a few representative examples. 

"Editing Function" means any computer-generated text editing operation, such as by way of example 
and not limitation, insert text, delete text, move text and substitute text. Some primary Editing Functions are 
listed towards the end of the description below. 

rs The preferred embodiment of a keyboard less entry computer system will now be described that 
includes a transparent input screen that generates positional information when contacted by a stylus, and a 
display screen mounted physically below the input screen such that a character that is displayed can be 
seen below the input screen. The system includes a computer programmed to compile the positional 
information into Strokes, to calculate Stroke characteristics, and then compare the Stroke characteristics 

20 with those stored in a database in order to recognise the symbol drawn by the stylus. Key features of the 
system are: (A) transparent position sensing subsystem; (B) underlying display on which to mimic drawing 
of sensed positions and to show characters or symbols; (C) means to convert sensed positions first into 
plotted Points and then into recognised characters or symbols; and (D) means to "learn" to associate 
sensed input positions with a character or symbol. The apparatus can be housed in a portable unit having 

25 dimensions up to about 400mm by 400mm by 100mm (16" x 16" x 4") and a weight of up to about 7 
kilograms (15 pounds), with a self-contained power supply. 

Brief Description of the Drawings 

30 The preferred embodiment of the invention will be described in more detail by way of example with 
reference to the drawings, in which: 

Fig. 1 is a schematic system block diagram of an embodiment of the present invention; 

Fig. 2 is a perspective view of the housing containing the operating elements of the system; 

Fig. 2A is an enlarged portion of Fig. 2, with parts removed to show the positional relationship between 
35 the touch input screen and the display screen. 

Fig. 3 is a schematic diagrammatic view of the input screen, stylus and associated electronics; 

Fig. 4 is an overall schematic system block diagram of the apparatus of a keyboardless entry computer 

system according to the present invention; 

Fig. 5 is a schematic block diagram depicting the movement of data within the system when modified by 
40 handwritten characters and commands; 

Fig. 6 is an overall system block diagram depicting the hierarchy of software used to operate the system; 

Fig. 7 is a generalized block diagram of the character and pattern recognition algorithm. 

Figs. 8A and 8B together are a detailed block diagram of the pattern recognition algorithm. 

Fig. 9 is a schematic block diagram of the Stroke characterization subroutine. 
45 Fig. 10 is a top plan view of a screen illustrating the "initializing" of the database for Handwritten 

Symbols. 

Figs. 11 A through 111 are a series of top plan views of screens depicting the operation of a text editing 
system. 

Figs. 12A through 12G are a series of top plan views of screens depicting the operation of a data entry 
so system. 

Fig. 13 is a generalized block diagram of the Linus Editor. 

Detailed Description of the Preferred Embodiments : 

55 With reference now to the figures, wherein like numerals indicate like elements throughout the several 
views, and in particular with reference to Fig. 1, an overall block diagram of a portable handwritten, 
keyboardless entry computer system 10 is depicted. The complete computer system is encased in a 
housing 12, indicated graphically by the dashed line, and includes a conventional, general purpose digital 
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microcomputer 14, described in greater detail hereinbelow. Input information is provided to microcomputer 
14 by stylus 16 "writing" on writing or input screen 18. Stylus 16 (Fig. 2) is connected to the computer of 
system 10 with wire 17 (Fig. 2). As stylus 16 "writes" on input screen 18, a plurality of locating signals 
representative of a plurality of corresponding positional coordinates are transmitted to microcomputer 14. 

5 Microcomputer 14 has been programmed in accordance with a computer program described hereinbelow, 
to recognize the stream of locating signals and to store these signals in a computer memory. The 
programmed microcomputer 14 also provides a corresponding plurality of display signals to a display 
screen 20. Both input screen 18 and display screen 20 are described in greater detail hereinbelow. 

Referring now to Fig. 2, there is shown a perspective view of keyboard less computer system 10 

w embodying the features of the present invention. Keyboardless computer system 10 is contained in housing 
12, which is a rectangular enclosed casing having a sloped top surface 22 with a multi-line, solid state 
display area 24. Input screen 18 is depicted in Fig. 2A as being positioned over display screen 20. In this 
example, display screen 20 displays a plurality of horizontal lines 25 with the following indicia: 

is Kane 

Address 



20 Handwritten entries are made above each line 25. The distance or space between two lines 25, denoted 26, 
is used by the system to normalize all distances, and lines 25 themselves serve as a reference axis or base 
line. 

Below display area 24 on top surface 22 is a key input section 26 comprised of a plurality of "Softkeys" 
28. Softkeys 28 can be programmed by the operator for any purpose, such as to enter computer 

25 commands. Exemplary commands for Softkeys 28 are "store, "recall," and "delete." In addition, Softkeys 
28 can be used to switch between different programs or between modes (e.g. data entry mode and edit 
mode). However, Softkeys 28 are optional and are used to supplement the input obtained by handwriting 
the entries. Stylus 16, used for writing input data and commands in display area 24, also is used to activate 
the selected Softkey 28. An ON-OFF switch 30 is positioned on the side of housing 12 adjacent to Softkeys 

30 28. A data output or peripheral connector 31 is located on the upper right side of housing 12. 

Input screen 18 can be a conventional resistive type touch screen in which a voltage is applied to the 
screen edges and a stylus detects the voltage at the touched location. The writing surface is a transparent 
material, typically glass, coated with a thin, uniform, conductive layer (presently, vacuum deposited indium 
tin oxide). Vertical bus bars or conducting strips (not shown) are used along the two sides to apply the 

35 reference voltage to determine the "X" coordinates of the stylus position and horizontal bus bars or 
conducting strips (not shown) are used along the bottom and top to apply the reference voltage to 
determine the "Y" coordinates of the stylus position. In this embodiment, stylus 16 is merely an electric 
probe that, when physically in contact with the conductive layer, detects the local voltage at the Point of 
contact, which will vary with the distance from the conducting strips or bus bars. With the origin at the Point 

40 of voltage application, the X, Y coordinates are inversely proportional to the impressed voltage. Stylus 16 
must make good contact to minimize adding resistance that would lower the voltage detected, and thus add 
an erroneous distance increment. In a presently preferred embodiment, a soft graphite tip is used. The 
voltage is conducted from the pen through a wire, such as wire 17 in Fig. 2, to an analog to digital converter 
for use in the computations described below. The stylus may be a charged "pen" as described herein, a 

45 light pen as is well known in the art, or any other hand-held device which can outline Handwritten Symbols 
on a screen. 

An example of a conventional electrostatic screen is disclosed in the aforementioned 4,318,096 Pepper 
patent. This resistive type screen has the advantage that the interference caused by the user's hand 
touching the screen is minimized. 

so Both horizontal and vertical position sensing is provided by alternately switching the voltage impressed 
on a conductive layer between the pairs of horizontal and vertical bus bars by an interface and multiplexer 
controlled by a microcomputer or microcontroller. In one commercially available input or touch screen, the 
bins bars are broken into a series of short strips with diodes to prevent the horizontal strips from shorting 
out vertical strips and vice versa. This technique is used in a commercially available touch screen from 

55 Touch Technologies of Annapolis, MD. and Elographics of Oak Ridge, Tenn. 

Referring now to Fig. 3, an alternate embodiment of a low power position sensing novel input screen 33 
is described in greater detail. Input screen 33 is also for determining an X,Y position on an electrically 
resistive plate 34. A stylus 35 containing a voltage source, such as battery 36 or a voltage transmitted to 



5 



EP 0 645 730 A1 



stylus 35 from an external source such as the system power supply, is used to touch screen 34 and apply a 
voltage at the touched position. When the touched position is charged by stylus 35 with a positive voltage 
with respect to a plurality of place measurement Points 37, the voltages at these Points will vary with the 
distance to the pen position, such as position Xi , Y t , indicated at 38. These voltages are sequentially 

5 measured in the X and Y directions by using conventional means, such as disclosed in the aforementioned 
prior art patents. In Fig. 3 these means are a conventional interface/multiplexer 42. A conventional Analog- 
to-Digital converter 43 converts the detected voltages into a digital signal. A microcontroller 44 receives the 
digital signal, performs standard checks to ensure the signal's numerical value is "valid" {e.g. is within the 
possible range of voltages), and then converts the voltages to X and Y distances in the manner described 

w herein. Microcontroller 44 is conventional, but could be replaced by a system computer. Microcontroller 44 
provides a digital signal representative of the X and Y distances to measuring Point 38 to an output port 46. 
Port 46 can be a conventional RS 232 port. Alternatively, microcontroller 44 could translate Point X t , Yi to 
any other reference Point, such as a Point on base line 25 (Fig. 2). 

As long as there is no contact by stylus 35 at position 38 or any other position on plate 34, no current 

is flows and power consumption is minimal. An incident measurement of the voltage at the measurement 
Points may occur by using ramped voltage at the positioning Point and timing when the measuring Point 
voltage exceeds a preset back voltage. 

There are various options for the input or touch screens 18 and 33: resistive plate 34 or its equivalent 
for screen 18 can be transparent or translucent and the position Point can be made by a stylus or a finger 

20 of the user, or a connecting Point of an overlapping conducive screen (such as the commercially available 
touch screens from Touch Technology Annapolis, MD.). Input screens 18 and 33 can be a physical solid 
surface which is transparent or translucent and can be glass or plastic such as Mylar. The surface can be 
coated with a conductive/resistive substance like indium tin oxide. Other physical surfaces can use sound or 
electromagnetic radiation transmission from the touched position to a reference Point or Points and the 

25 distance is determined by the time delay or phase shift. Alternatively, input screens 18 and 33 can use an 
ethereal or geometric surface defined by an electromagnetic, optical or sonic field. 

Position detection can be accomplished with electrical contact closure by resistive, capacitive or 
inductive coupling, remote sensing by sonic, electric or magnetic fields or by light (UV, IR, or microwave) 
scanning. 

30 The advantages of the low power position sensing input arrangement over other such screens are: 1) it 
makes stand-by power requirements minimal; 2) it eliminates distortion due to opposing parallel "bus" bars 
in conventional touch screens; and 3) when a ramped voltage is employed, it eliminates the need for an A/D 
chip which is a major cost factor in state-of-the-art touch screen technology. 

The coefficient of friction of the screen 18 is desirably selected to be "rough" enough to offer some 

35 resistance to the movement of stylus 16 on the screen. If the screen were too smooth, the stylus would 
slide too easily and would be difficult to control. 

Reference is now made to Fig. 4 which discloses an overall system block diagram of the major 
electronic circuitry used in the preferred embodiment of the present invention. Microcomputer 14 includes a 
microprocessor 50, interconnected to a plurality of other electronic elements by means of data path or bus 

40 52. A Read-Only-Memory (ROM) 54 which is programmed with the operating and application programs and 
a battery powered Random Access Memory (RAM) 56 is connected for bidirectional data flow onto bus 52. 
Microprocessor 50 may be a conventional single-chip eight-bit or sixteen-bit device which functions to 
execute the fixed control programs residing in ROM 54, and further receives control programs from and 
provides control signals to the other electronic elements via bus 52. Microprocessor 50 may be of the type 

45 commercially designated Z80 (manufactured by Zilog Microcomputers of Cupertino, California), of a type 
8088 device (manufactured by Intel Corp. of Santa Clara, California) or any similar or more powerful 
microprocessor. ROM 54 may be of the type 2564 or 4764, both manufactured by Texas Instruments of 
Dallas, Texas. The storage capacity of RAM 56 is determined in part by the sizes of the application 
programs, the operating program and the database. As discussed below, RAM 56 may be of the static 

so SRAM or dynamic DRAM type. The primary requirements of RAM 56 are that it have sufficient storage 
capacity and that it require a minimum of input power. 

A battery 58, such as a lithium battery, provides power for making the memory of RAM 56 non-volatile 
for extended periods of time. A battery pack 60 containing the well-known rechargeable types of batteries is 
used to provide the various voltage levels required by the other electronic elements of microcomputer 14. 

55 Alternately, the storage function of RAM 56 may be served by a non-volatile device which requires no 
power for maintaining storage, such as an electronically erasable and reprogrammable memory (EE PROM), 
or devices using magnetic bubbles or capacitance. State-of-the-art disk or tape may also be used for mass 
storage. Suitable bubble memory devices include types 7110 and 7114 which have storage capacities of 1 
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megabit and 4 megabits respectively. (Both are manufactured by Intel Corp.). Furthermore, it is possible to 
use a single integrated circuit chip which includes microprocessor 50, at least part of ROM 54 and at least 
part of RAM 56. 

Also connected to bus 52 is an EIA RS-232 serial interface 62 which provides a means for inputting and 
s outputting data. Data is provided to bus 52, (usually to RAM 56) by interconnecting an external data source 
to RS-232 port 62 directly to the microprocessor 50 and other elements of the microcomputer 14. 
Offloading data from RAM 56 can also be done by microprocessor 50 to an external computer, other data 
gathering device, a mass data storage device (e.g. floppy and hard disk drives) or an electronic 
telecommunications system. In like manner data can be communicated through port 62 to a printer (not 
io shown) from interconnecting bus 52. 

Stylus 16 is used to write on input screen 18 and to cause the generation of X,Y coordinate information 
by conventional touch screen interface electronics circuitry. The coordinate information is communicated via 
the bus 52 for control use by system 10. The solid state display 20 consisting of a multi-line display -- 
illustratively 80 columns by 25 lines - is interconnected to bus 52 through a display interface 66. The 
75 fundamental requirements for the display are that it be substantially flat and sufficiently thin for use in the 
present invention. The display may be of the following types: scanning types such as a cathode ray tube, 
projected types such as a rear-view projector, light emitting array of Points types (e.g., electroluminescent 
or plasma discharge) and light blocking array of Points types (e.g., liquid crystal displays, solid state PLTZ 
or magneto-optical). In addition, it is preferable that the display be compatible with input screen 18 in size, 
20 configuration and transparency, and that both be low power consuming types. 

The X,Y coordinates are input to keyboardless computer 14 via input screen interface electronics 64 
and communicated via bus 52 to microprocessor 50 which executes programs stored ROM 54 and RAM 56. 

The number of Points (Le., sets of X,Y coordinates) used in defining each Handwritten Symbol and the 
speed at which Points are identified are important for practical utility. It is desirable to use at least about 4 
25 Points per millimetre (100 Points per inch) and at least about 100 Points per second to define Handwritten 
Symbols. It is to be noted that the more Points per unit length that are identified, the greater the accuracy 
of the system in identifying Handwritten Symbols - however, more Points being identified will slow down 
the speed of identification and require more computer memory. Accordingly, a balance will have to be 
achieved, based on the size (available memory and processing ability) of the computer system and the 
30 requirement for speed of response and accuracy. For most purposes, standards in the range from about 4 
Points per mm (100 Points per inch) and 100 Points per second to about 8 Points per am (200 Points per 
inch) and 200 Points per second will be suitable. 

It is also to be noted that the greater the precision of the system in identifying the X,Y coordinates of 
each Point the fewer the number of Points needed to be identified per unit length and per second to 
35 accurately identify Handwritten Symbols. Conversely the less the accuracy, the more Points that are 
needed. 

Point resolution is needed to place Points where intended, e.g. , to write an editing symbol precisely 
between the two characters. Ideally, resolution to a single display pixel is desirable. However, operationally, 
resolution within two displayed pixels is sufficient for a display with 640 pixels in a 225mm (nine inch) 
40 horizontal scanline. 

When switch 30 (Fig. 2) is positioned to "power on", the basic display mode is active and microcom- 
puter 14 (Fig. 4) programmed by the operating system, causes a menu to be displayed on display screen 
20 (Fig. 1). The menu presents various software options. A primary software function, editing, functions in a 
manner similar to conventional word processing software with the difference being that handwritten 

45 characters, symbols and commands are interpreted by the system as if they were entered from a 
conventional keyboard. The system is capable of learning the editing symbols used by a particular writer for 
functions such as indent, insert, delete, move and reformat and translates those symbols into digital 
command functions. Optionally, Softkeys 28 (Fig. 2), activated by touching those areas on the input screen 
with stylus 16, function like conventional hard function keys on a computer keyboard. 

50 The system is particularly adapted for use as an interactive screen editor or word processor. After a 
writer retrieves a document by (for example) touching the displayed name of an existing file with the stylus 
or by writing the name of the file on the screen, all usual editing functions can be performed with stylus 
entry. When the user wishes to change a displayed character or symbol, he may simply write over the 
displayed character or symbol and as described hereinbelow the pattern recognition algorithm will translate 

55 the written entry into computer text. For example, the editing software allows text to be eliminated by simply 
drawing a line through it and a conventional caret symbol may be used to change the operating mode to 
the insert mode. In the insert mode, display screen 20 provides additional space for entry of handwritten 
characters or symbols which are inserted in the text after the Point where a caret was written in. Text can 
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be moved simply by placing brackets or other user-defined delimiters around a displayed phrase or word 
and writing a caret or other user-defined symbol in the area of the text in which the user wishes this 
material to appear. New margins can be set by drawing vertical lines down the side of the displayed text 
where the new margins should appear. 
5 The basic editor software also allows new documents to be created by simply writing Handwritten 
Symbols on the screen. All documents can be stored, changed and communicated in the manner in which 
these functions are accomplished on a conventional word processing system with the difference that these 
functions are accomplished with handwritten Editing Symbols on the (optional) screen or by touching the 
Softkeys with the stylus. The composite text thus produced and stored can be subsequently offloaded 

w through the RS 232 port 62 (Fig. 4) to another computer, a similar device, an external data gathering device 
or recording device, into a printer or through a telecommunications system. 

In addition to these major operating modes, a number of ancillary elements and features add to the 
utility of the system. A conventional alphanumeric keyboard (not shown) containing a full set of keys can be 
connected to a conventional keyboard interface (not shown) to support the entry of alphanumeric char- 

75 acters. An AC/DC power connector may also be used in those applications when portability is not needed 
and when needed to meet the power requirements of screen technologies such as gas plasma displays and 
electroluminescent displays. 

In actual use the keyboardless computer can function in any application or environment in which 
handwritten input translated into computer text is useful or necessary. Illustratively, the device can function 

20 as a new generation word processor, or for use in fields such as sales, nursing, inventory control, census 
taking, claims adjusting, to name just a few of its many uses or as a general learning and testing device in 
education. Since the pattern recognition software can learn and translate into computer text from languages 
which are not made up of a small or limited set of alphanumeric characters (e.g., Japanese, Korean, and 
Chinese), it has particular utility for word processing and telecommunications in these languages. 

25 In practice, it is particularly desirable to use a single computer screen to display any initial forms, Font 
Symbols or other displays to be edited, and to create a nearby "window" of blank space where Handwritten 
Symbols are to be written, displayed and identified, and where the Font Symbols corresponding to the 
Handwritten Symbols are to be displayed. In this way, the user can view the text being edited and the 
proposed insert or change without significant movement (if any) of the head and eyes. This is illustrated in 

30 Figs. 11A to 11D. This feature (proximity on one screen of text to be edited and the window into which new 
text is to be handwritten) is very important to the simple, rapid, comfortable use of the system. 

In a preferred embodiment of this invention, the system "learns" the handwriting of a particular user 
prior to actual use. For example, if using the Roman alphabet, the twenty-six letters of the alphabet and the 
numerals from 0 to 9 would be inserted into the database. Punctuation symbols, such as periods, commas, 

35 question marks, colons, semi-colons, hyphens and the like could also be inserted. There is virtually no limit 
to the Handwritten Symbols which can be recognized and stored in the database. Of course, the computer 
will have to store a suitable array of Font Symbols for conversion of the Handwritten Symbols. Different sets 
of Font Symbols could be created and stored in the permanent memory of the computer, as in ROM chip 
54. For example, in English language usage, a chip could contain one (or more) fonts of numbers and 

40 letters, suitable punctuation symbols and appropriate mathematical symbols. Other chips could have stored 
Font Symbols for the Arabic, Cyrillic or Greek alphabets, Japanese, Chinese or Korean "Konji", symbols for 
use by architects or engineers, or chemical symbols ( e.g. , benzene rings and the like). 

In Fig. 10, one of a series of learning screens is displayed and the user is prompted to write the 
numbers 0 through 4. The computer will attempt to match the written numbers with the existing database (if 

45 any). If it cannot be matched because there is no existing database or because there is a poor match with 
an existing database, the character is added to the database. This learning process continues until all of the 
alphanumeric (or other) characters and symbols to be used are entered into the database. The system has 
the capability of storing multiple Stroke characterization databases for systems used by more than one 
user. The existence of a unique Stroke characterization database for each user has the further advantage of 

so making the writing angle irrelevant. As a result, the system is adaptable to all handwriting styles and is 
usable by right-handed and left-handed persons. One feature may desirably be incorporated into the 
apparatus to accommodate left-handed and right-handed persons. This feature is a receptacle (not shown) 
for the stylus connector on either side of housing 12, so that the stylus 16 may be connected on the left 
side for left-handed persons and or the right side for right-handed persons. 

55 Fig. 10 also provides an example of the use of "Softkeys". In addition to the input line, a variety of 
Softkeys appear. Each Softkey corresponds to a function that can be performed by the system. In order to 
execute the function, the user merely touches the indicated Point with the pen. The Softkey will then appear 
in reverse video and the selected function is performed. There are numerous advantages to Softkeys over 
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traditional function keys. Some of the more significant of these are that the user is no longer required to 
memorize what function key performs what function; the need for keyboard overlays is eliminated: and 
different Softkeys can be made available (displayed and made operational) at different Points within a 
program. 

5 Figures 11 A to 111 demonstrates some of the simplifications in word processing made possible through 
the use of this invention. In Fig. 11A a standard screen of text is displayed. The user of the keyboardless 
entry system decides that additional information needs to be added and draws an insert symbol ( e.g. , caret) 
on the screen at the desired position. A data entry "window" then appears. (Fig. 11B). The text is written in 
as Handwritten Symbols (Fig. 11C), matched (converted to Font Symbols) (Fig. 11D), and then inserted 

70 (Fig. 11E). The operator reconsiders the addition and draws a horizontal line through the new material. (Fig. 
11F). It is immediately erased. (Fig. 11G). Next, the operator decides that a larger right-hand margin would 
be more appropriate for the text. A vertical line is drawn on the screen (Fig. 11H) and the margin is 
automatically adjusted (Fig. 111). 

A generalized block diagram of the editing process is provided in Fig. 13 and a description of that 

75 figure appears here inbe low. 

Figures 12A-12G illustrate how a blank form may be used for a hospital patient. The user of the system 
first calls up the proper blank form (Fig. 12A). This may be done, for example, by touching an appropriate 
Softkey. The area where the information, in this case a pulse observation, is to be inserted is touched with 
the pen (Fig. 12B). After the desired location is highlighted, a "window" appears directly below the space 

20 where the observation is to be recorded (Fig. 12C). The nurse then touches the pen on the match box 
which appears highlighted (Fig. 12D). The software then matches the handwritten input to the corresponding 
Font Symbols and displays the result (Fig. 12E). If there is an accurate match, the "insert" block is touched 
(Fig. 12F), and the new observation is added to the patient's records (Fig. 12G). This mechanism is clearly 
applicable to a wide variety of "blank forms" in which data is inserted into a form or corrected. For 

25 example, it could be used to correct or update financial information in a spreadsheet program. Other 
information can be recorded in the same manner. 

The reason for using a black background and white letters for the newly entered Font Symbols is to 
facilitate checking the accuracy of the inputted character. Although this is preferable, it is not essential and 
a white background and black letters is also acceptable. 

30 The ability to create a window and input data on the same screen and in physical proximity to the text 
being edited or space for data to be input is an important feature of the apparatus, for it permits ease and 
speed in the use of the apparatus. The user's eye may focus on the space where the data will be inserted 
and the ability to contemporaneously display Handwritten Symbols and the corresponding Font Symbols 
makes it easier to see errors, when the system "misreads" a Handwritten Symbol, and then corrects errors 

35 quickly and easily. 

Referring first to Fig. 5, the overall operation and functioning of the pattern recognition software will now 
be described. When the operating system calls the pattern recognition program, the program begins in 
terminal 75 where a number of variables and counters are initialized. The software then proceeds to 
decision diamond 76 where the program determines if stylus 16 (Fig. 2) is in contact with input screen 18 

40 (Fig. 2A). The system provides a "pen down" signal, as shown in processing box 78, as well as the X,Y 
coordinate voltages as locating signals, as described above. Microcomputer 14 (Fig. 4) converts the X,Y 
coordinate locating signals into Stroke characteristics using programs stored in ROM 54 (Fig. 4), or a 
separate microcomputer can do the conversion, such as microcontroller 44. If a pen down signal is 
received, the software proceeds to processing box 80 where the individual locating signals are combined 

45 into "Strokes," a Stroke being defined as the Point locating signals produced between a "pen down" signal 
and a "pen up" signal. 

The system then calculates a transform, as described below, for each Point transforming the Point 
coordinates from the X,Y cartesian coordinate system to a relational coordinate system. The software next 
proceeds to processing box 82, where it compares the Stroke with previously entered Strokes accumulated 

so into a database, and determines if the Stroke is represented by a symbol in the database. If a match is 
found (if the Font Symbol represented by the Strokes is recognized), as indicated in decision diamond 84, 
microprocessor 50 (Fig. 4) causes the symbol to be sent to display screen 20 (Fig. 4) as indicated in 
processing box 86. If a match is not found, microprocessor 50 (Fig. 4) causes a message to be displayed, 
as indicated in processing box 88, which requests further input from stylus on input screen 18 (Fig. 4) by 

55 either flashing an entry which is close to a match or a non-recognition symbol. 

As mentioned above, the software compares the Stroke characteristics of each handwritten Symbol to 
data entries previously stored in a database. In a preferred embodiment, the database is arranged into 
sections of characters or symbols by the number of Strokes needed to make the character or symbol. 
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Within each section, the entries are randomly arranged at first, but after use, as explained herein, the most 
frequently used entries "rise" to the top of the database. It should be noted that each user will have his or 
her own particular style of writing a Handwritten Symbol and that each Handwritten Symbol may have a 
number of different variations. 
5 For example, many people write the lower case letter "h" using a single Stroke. They do this by 
starting the pen on the writing tablet at a Point where they wish to place the top of the letter, drawing a 
vertical line downwardly to the base line, then without removing the pen from the paper, proceeding back 
up to the midPoint of the previously drawn vertical line, over to the right and down to the base line when the 
pen is picked up from the paper. On the other hand, these same people may draw the upper case letter 

w "H" using two Strokes. They do this by drawing the left hand vertical line and horizontal line as is done for 
the lower case "h\ picking the pen up from the tablet, and then drawing the right hand vertical line. 
Appendix I displays the data of the Stroke data Points for these two letters as the data is stored in memory 
after having been generated by an embodiment of the present invention. 

As shown in Appendix I, the letter "h" as drawn at one particular time by one user has one Stroke <ns 

75 =1) with 20 Points (np = 20) and x and y coordinate characteristics for the minimum, mean and maximum 
normalized values (1/80th of a line width) as follows: -17 and -6; 0 and 18; and 19 and 60, respectively. The 
values in the first vertical column are the Point-to-Point slopes, normalized to 360°/256. The values in the 
second vertical column are the Point-to-Point average vertical positions above the base line, normalized to 
1/80 of the line width. A typical line width is about 1mm (0.04 inches). 

20 Referring now to Fig. 6, a software hierarchy of programs is depicted. At the top, overseeing the entire 
operation of computer system 10 (Fig. 1), is an operating system as indicated by box 90. Applications 
programs shown in boxes 92 and 94, residing in RAM 56 (Fig. 4) and ROM 54 (Fig. 4) can be executed by 
microprocessor 50 (Fig. 4) under control of the operating systems. When a Handwritten Character is 
required or is indicated by an interrupt, handwriting recognition software 96 is called. A first subroutine, 

25 indicated in box 98, encodes the X,Y coordinates into Strokes. The characteristics of the Strokes are then 
defined by a subroutine 100 followed by comparisons of the Strokes with a database that has been loaded 
from ROM 54 (Fig. 4) into RAM 56 (Fig. 4). The comparison is made by a subroutine 102. When the 
operating system is in the "learning" mode, the database is updated with the new Stroke data and symbols, 
as indicated in box 104. Similarly, a previously stored document can be edited by applications program 92 

30 by using edit function 94 as called by the operator, who provides the instructions as input using the 
subroutines 98, 100 and 102 of handwriting recognition program 92. 

Referring now also to Fig. 7, operating system 90 (Fig. 6) executes the Handwritten Character 
recognition software 96 (Fig. 6) by accepting as input the X,Y coordinate Points, depicted in box 110, of the 
position of stylus 16 (Fig. 2) on input screen 18 (Fig. 2) and encodes these Points into Strokes as depicted 

35 in box 112. The program then characterizes the Strokes by some description set, such as considering the 
length, curvature, slope, and position of the Stroke, as depicted in box 114. In box 116 the best comparison 
is then found of the characterized Stroke or sequence of Strokes with those in the database. If a sufficiently 
close match is found, the character is identified in box 118 and the database entry is swapped with the 
entry above it as shown in box 120. In this way, the most frequently identified characters will "rise" to the 

40 top of the database and the overall system performance, as measured in time to find a match, will be 
increased. If a match is not found, the user can add to the bottom of the database, as indicated in box 122. 

With reference now to Figs. 8 and 9, a flowchart of the computer program to recognize a particular 
Stroke sequence is set forth. The computer program begins in terminal 150 and proceeds to process the 
X,Y voltages from processing box 152, the voltages having been converted to a digital signal. The program 

45 then proceeds to decision box 154 where the program determines whether the pen or stylus 16 (Fig. 2) is 
out of contact with input screen 18. This determination is made by both the X voltage and the Y voltage 
being zero. If the program determines that the pen is up, then the Stroke is determined as having been 
completed and the program branches to decision box 156. In decision box 156, the program determines 
whether there are less than three Points in the Stroke and if so the program branches to decision box 158. 

50 In decision box 158, the program determines whether there are zero Points in the Stroke. If there are zero 
Points in the Stroke, then the program loops back to the beginning of processing box 152 where another set 
of Points is read. If the Point counter (incremeted in processing box 164) indicates that there are more than 
zero Points, the program branches to processing box 172. In processing box 172 the Stroke is identified as 
a dot and its height above the base line (HABL) is calculated in processing box 173 . From process box 173 

55 the program proceeds to processing box 171. 

However, if the pen down signal is received, the program branches to processing box 160 where the 
voltages are scaled to determine the coordinate Point using the following formulas: 
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X = aivi + bi 
Y = a 2 v 2 + b2. 

The constants ai and bi are scaling parameters that are determined from calibrating the input surface of 
5 the particular display. 

Once the voltages are scaled, the program proceeds to decision diamond 162 where the program 
determines whether it is an erroneous Point. This is done by comparing the distance between Points and 
eliminating a Point if the distance is too great (greater than 2.5mm or 0.10 inches is presently used). On the 
other hand, a Point is also eliminated if the Points are too close together. Points are presently thinned out if 
70 they are within about 0.4mm (0.015 inches). 

The comparison problem that exists for the first Point is resolved by determining if a Point is the first 
Point after a pen is down and then that Point is used only to check the next Point which is accepted, 
assuming that that Point is within the maximum distance of 2.5mm (0.10 inches). 

If the distance between Points is determined as being outside the two criteria, the program drops the 
75 Point and branches back to the top of processing box 152 to read another pair of coordinate Point voltages. 

On the other hand, if the Points fall within the criteria, the program continues to processing box 164 
where a Point counter is incremented to keep track of the number of Points. This number is used in 
decision diamond 156, as mentioned hereinabove. The program then continues to processing box 166 
where the Points are smoothed according to any one of a number of formulae. Smoothing is used to 
20 minimize noise from digitization, from erratic hand motion and from electronic noise. The simplest 
smoothing technique is a multiple Point average which results in calculating new Points (x/y/) as follows: 



25 



<j f = I x i /(n 2 - n 2 * i) 



30 And similarly for Vj, smoothed over Points ni - n 2 . 

Another simple method is called the running weighted average method and utilizes the following 
formula: 

Xj f = a Xj + (1 - a) Xj* -1 

35 

Alpha is a weighting constant that is usually positive (and less than one) and has been used at 0.25. The 
summations have been taken with n 2 minus ni equal to one. A third method involves what is called a spline 
fit wherein the following formula is used: 

40 Xj' = (Xj- t + 4Xj + X J+t ) 

Any of the foregoing methods can be applied either before or after filtering. The filtering is done so as 
to reduce the number of input Points and to space data so that difference and/or angle calculations can be 
made within acceptable random error bounds. A simple process of thinning a sequence of Points by 
45 excluding the acceptance of subsequent Points within a set distance of the previously accepted Points has 
been found to be an effective filter. 

From processing box 166, the program proceeds to processing box 168 where the Point is stored in an 
array that is incremented for each new Point since the last pen down signal. Thus, an addressable array of 
Points is created for each sequence of Points obtained from a pen down to a pen up signal. This sequence 
so of Points is called a Stroke. From processing box 168, the program loops back to the top of processing box 
152 where another Point is obtained until a pen up signal ends the Stroke. 

In decision diamond 156, a determination was made as to whether there were less than three Points in 
a Stroke. By definition, if there are three or more Points in a Stroke, the Stroke is a line and not a dot. If 
there are three or more Points in the Stroke, the program branches to subroutine box 170. In subroutine box 
55 170, discussed in greater detail hereinbelow with respect to Fig. 9, the Stroke is characterized as to its 
slope and base line height. 

As can be seen from the foregoing, the segmentation of the stream of coordinate Points into a Stroke is 
bases primarily on the determining when stylus 16 is "up" or not in contact with the surface of input screen 
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18. Alternatively, a stream of Points can be segmented to form Strokes on the basis of other considerations. 
For example they can be segmented based upon changes in a locally calculated curvature or upon a large 
local curvature. Local curvature is calculated by the change in distance along the input coordinates divided 
into the change in slope. This produces radius of curvature. When the radius of curvature changes rapidly 

5 with respect to distance along the input coordinates, or if the radius is too small, then a segmentation 
Stroke is assumed to end, and a new Stroke begun. Further segmentation techniques can look at the 
relative maximum and minimum in one or both coordinates and/or the curve crossings in the coordinates. 
However, these latter two methods have been determined to be less effective. 

Characterizing a Stroke reduces the sequence of coordinates defining the Stroke or segment to a set of 

10 characteristics that are unique, generalized and minimal. Uniqueness refers to both factors that the same 
characteristics are generated by the same coordinates and that the characteristics are sufficient to 
regenerate an approximation to the original coordinate sequence. The term "generalized" is used to mean 
that the characterization is invariant under such transformations so that the symbols are invariant (e.g., 
translation and scaling or stretching or small tilt). The scaling of all distances is accomplished by taking a 

75 ratio of the distance to a writing entry line width. 

The minimal set of segment characteristics have the following features: 

(1) Stroke position: one or more of centroid/average, extent extreme or beginning and ending Points 
determined relative to the writing entry line, to previous Strokes, or to character extent or center; 

(2) Stroke shape is characterized by one or more of average slope, change in slope (which is a measure 
20 of average curvature) and/or a change in curvature, by sequence of slopes over specific length segments 

or over fractional lengths, or by a gross description of linear direction or circular completion and opening 
direction; 

(3) And Stroke length as characterized by distance along the curve and/or the extent extremum along the 
coordinate system. 

25 In one embodiment of the present invention, positioning by centroid, extent extremum, and starting and 
ending coordinates have been successfully used. The Stroke shape is encoded as a sequence of slopes 
and vertical positions (relative to Stroke centroid). The Stroke length can be approximated by the number of 
filtered Points. Alternatively, the average curvature can be encoded in total slope change (along with 
length), change in starting to ending slope or fitting the slope angle versus length curve for rate of change 

30 of slope angle. Additional characteristics that could be used include location of coordinate relative extrema, 
curve crossing, cusp, and Stroke direction. A particular method used to determine the unique characteristics 
is set forth below. 

1. The numerical values of the Criteria for each Stroke of the Handwritten Symbol are determined. 

2. The database values for each stroke of the previously learned Handwritten Symbol is determined and 
35 subtracted from the newly determined values respectively. 

3. The absolute values of each difference are scaled, to make each of the five measurements reasonably 
equivalent to the others such as lengths scaled to height between lines. 

4. The five thus-determined values are added. 

5. A predetermined threshold is used as "goodness" test of recognition - too high a value and Font 
40 Symbols are infrequently recognized and too low a value causes Font Symbols to be misidentified. 

Thresholds of approximately 1,000 are used initially and then switched to approximately 100 for 
improved recognition. If the threshold is exceeded, the comparison is discarded and an error message is 
created and displayed. 

6. The database is searched to find a numerical minimum difference. If the minimum difference is below 
45 the acceptable threshold for recognition, the corresponding Font Symbol is displayed on the screen or 

the command is performed, as the case may be. 
It has also been found that the preferred classification of a Stroke is a continuous one, rather than one 
that is grossly discrete. For example, determining a slope by angle in 256 directions rather than in 8 is 
desirable. Other non-continuous classifications can include bars/arches/hooks, number and closure of cusps 
50 or horizontal or vertical Strokes. 

From subroutine 170, the program proceeds to processing box 171 where both an individual Stroke and 
one or more preceding Strokes are compared with a database entry that is stored in RAM 56 (Fig. 4). 

This comparison initially begins with three eliminating questions that are asked by the program in 
decision diamonds 174, 176, and 178. In each case, if the database entry is eliminated, the program 
55 proceeds to a processing box 180 where the address of the next data entry is received and from which the 
program loops back to the top of processing box 171. In decision diamond 174, the first eliminator is asked 
by seeking if the number of Strokes are different. If the number of Strokes are the same, the program 
proceeds to decision diamond 176 where the average Height Above Base Line (HABL) is calculated and 
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compared with the HABL of the data entry. The entry is eliminated if the difference in the average HABL's 
is greater than one-half the height of the entry line. From a negative determination in decision diamond 176, 
the program proceeds to decision diamond 178 where the number of Points per Stroke are compared and 
the database entry is eliminated if the difference in number of Points is greater than ten. This determination 

5 varies from that made in decision diamond 174 because it is concerned only with the number of Points for 
each Stroke. However, in decision diamond 174, certain letters, such as the capital letters "E" and "A", 
have more than one Stroke per letter. 

If a data entry is not eliminated by decision diamond 178, then the program proceeds to processing box 
182 where the program calculates a gauge to be used to determine the closeness of the match between the 

70 selected entry in the database and the drawn Stroke. A presently preferred gauge is the sum of the 
absolute values of the differences between the Stroke values and the database entry values of: 

a) distances or lengths in units of 1/80th of the line height (e.g., space 26, Fig. 2); and 

b) the slopes in units of 1/256th of 360° over all the Points along the diagonal of the comparison matrix. 
Alternatively, Dynamic Programming Techniques can be used to optimize the comparison using off-diagonal 

75 elements as well. 

From processing box 182, the program proceeds to decision diamond 186 where a match is 
determined. In actuality, a match is determined by the application of an arbitrary gauge (maximum 
allowable variance), which is the sum of absolute values of the differences between the entered Stroke 
characterization and that of the stored database entry. In processing box 183, the lower of the present 

20 gauge and the previous lower gauge is saved as the best match. The program then goes to decision 
diamond 184 where a determination is made whether the present entry is the last database entry. If it is not, 
the program branches to processing box 180 where the next entry is selected. If it is the last entry, the 
program proceeds to decision diamond 185 where a determination of a match is made on the basis of the 
gauge being below a predetermined threshold. This threshold is set by the user based on experience with 

25 the system. 

If no match is obtained, the program branches to decision diamond 188 where a determination is made 
whether all Strokes have been checked. If the last Stroke has been checked, then the present Stroke is 
compared in sequence with a previous Stroke to all two Stroke entries. As in the comparison with all one 
Stroke dictionary entries, the best fit comparison for all entered Strokes is the recognized symbol or 

30 sequence of symbols. 

However, if the last Stroke has been read and there still is not a match, then the program proceeds to 
processing box 190 where a question is displayed on display screen 20 asking the user if a new Font 
Symbol should be added to the database. The user responds and that response is used in decision 
diamond 192. Either the Stroke sequence is added to the database in processing box 194 and the program 

35 branches back to the top of processing box 152, or the program branches immediately to the top of 
processing box 152. 

On the other hand, if a match is determined in decision diamond 186 the program branches to 
processing box 195 where the programs shuffles the database by interchanging the serial location of the 
matched entry with the entry above it. The program then proceeds to processing box 196 where the 
40 program zeros the Point counter and the increment counter. The program next proceeds to processing box 
198 where the matched and characterized Stroke or Strokes are displayed by the computer as the identified 
Font Symbol. This display is located at the position in which the entry was made on input screen 18 (Fig. 
2). 

From processing box 198, the program proceeds to processing box 200 where the program can act on 
45 any commands which it has interpreted. An alternative characterization of the Stroke uses the Points 
themselves rather than the length, scope, curvature and position. 

With reference now to Fig. 9, this Stroke characterization is depicted in greater detail. Stroke 
characterization subroutine 170 essentially performs a mathematical transformation of each Point on a 
Point-by-Point basis to transform the Points from an X,Y Cartesian coordinate system to one in which the 
so coordinates are the normalized slope of each Point and the normalized height of each Point above the base 
line (HABL). 

Subroutine 170 first calculates the Point to Point slope in processing box 220 and then calculates the 
height of each Point above the base line in processing box 222. The slope and HABL of each Point are then 
normalized respectively to 1/256th of 2 Pi and to 1/80th of the width of the entry line in processing box 224. 
55 From processing box 224, the system proceeds to processing box 226 where the calculated normalized 
values for each point are stored in an addressable array. The subroutine then returns to the program 
through terminal 228. 



13 



EP 0 645 730 A1 



When the comparison is made between each Stroke and the stored values, the comparison is made by 
the normalized Point slope and Point height above base line. As mentioned above, a match is determined 
by an arbitrary gauge which is the sum of absolute values of the differences between the written Stroke and 
the stored or dictionary Stroke. The system learns by adding new Strokes to the dictionary database. Once 
5 the database fills up, those Font Symbols that are infrequently used are replaced by new entries. 

In a working embodiment of the present invention, the algorithm successfully identified upper and lower 
case letters and numbers when written discretely from one another. For Handwritten Symbols that are 
written such that they are continuous, direct extrapolation would require searching a database sequentially 
for one, two, three, etc. Stroke symbols and looking for the best fit. Upon identification of a Stroke fit, a 
w "new" letter is tenatively recognized, except that the next few Strokes are analyzed to check if they change 
the previous symbol for a better fit. For example, two Strokes that have been identified as "ones" would be 
combined and changed to the capital letter "H" once a cross bar was identified. 

The system design demonstrated by Figs. 7 to 9 could easily be coded by one with ordinary skill in the 
art of computer programming into almost any computer language. The source code listings for one 
75 application program utilizing the disclosed invention is included as Appendix II. The software in Appendix II 
is written in Microsoft Basic, a common computer language available for virtually all microcomputers and 
operating systems. The program is a complete text editing demonstration, which shows the improvements 
that can be made upon traditional word processing systems. 

Program lines 2600 to 4000 contain the character recognition subroutine which includes the software 
20 code necessary to get X and Y coordinates. This section of the program corresponds to Fig. 8. 

Program lines 2600 to 2699 make up a subroutine designed to obtain the X and Y coordinates of a 
given Point. This code corresponds to boxes 152, 154, 160, 162, 164, 166 and 168 of Fig. 8. 

Program lines 3000 to 3339 constitute a Point and Stroke analysis and characterization routine 
embodying boxes 156, 158, 170, 172 and 173 of Fig. 8. 
25 Program lines 3700 to 3790 make up a subroutine designed to compare the analyzed Strokes to a 
Stroke database. These program lines embody boxes 171 to 184 of Fig. 8. 

Program lines 3810 to 3980 make up a subroutine which is designed to learn a new character. This 
code embodies boxes 186, 188, 190, 192 and 194 of Fig. 8. 

Program lines 3060 to 3273 make up a subroutine designed for Stroke characterization purposes. This 
30 section of the code corresponds to Fig. 9. 

Program lines 3060 to 3095 are used to calculate Point-to- Point slopes and embody box 220. 

Program lines 3058, 3241 and 3262 are used to calculate a height above baseline (HABL) and 
correspond to box 222 of Fig. 9. 

Programs lines 3253, 3270-3273 are used to normalize the Point height ad slope and correspond to box 
35 224 of Fig. 9. 

Program line 3253 is used to store the height above baseline an embodies box 226 of Fig. 9. 
The foregoing program can be stored in the memory of a microcomputer of microprocessor with a 
requirement of approximately 25K of machine memory, so that it can be seen that the use of the program 
does not use up a lot of expensive memory and is relatively fast in executing the program's operation. If the 
40 program is written in a language other than Basic, requiring less memory, such as assembly language, the 
size of the program can be made smaller. 

Boxes 195 to 200 of Fig. 8 appear in logical places throughout the code. 
A dictionary of the variables of the relevant code section is included as Appendix III. 
With reference now to Fig. 13, a flowchart for the editing software ("Editor") demonstrated by Figs. 11 A 
45 to 111 and described above is depicted. Once the Editor is loaded into the system (box 229), control of the 
screen is returned to the system. The system then proceeds in the normal manner described above to 
acquire Points and display them (box 230), convert the Points into Strokes (box 231), characterize each 
Stroke (box 232), and attempt to match the Stroke or Strokes with the database (box 233). In processing 
box 234, the system sends each Handwritten Symbol to the Editor to interpret and execute a command if 
50 necessary. At decision diamond 235, the Editor determines whether the Handwritten Symbol is an Editing 
Symbol or a Font Symbol. If the character is determined to be an Editing Symbol,the Editor proceeds to 
processing box 236 where it determines which Editing Symbol has been entered and executes the Editing 
Function. If the character is determined not to be an Editing Symbol, then the alphanumeric character 
corresponding to the handwritten entry is displayed at processing box 237. In an alternative configuration of 
55 the Editor, Font Symbols will only be accepted when the Editor is in the "insert Mode." This structure 
insures that each Font Symbol is verified before being added to a document. 

The Editor uses a variety of symbols designed to make editing on the system similar to, but much 
more efficient than, traditional editing with pencil and paper. These primary editing functions include, but 
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are not limited to: 

DELETE symbol - n " A horizontal line drawn through a character or characters. The Editor will 

remove the underlying characters and reformat the text. 

ADJUST MARGINS symbol - "|" A vertical line longer then the height of one line on the display. The 
5 Editor will adjust the margin to the indicated position and reformat the text. 

INSERT symbol - "A" A caret drawn at the Point where text is to be added. The Editor displays an 
input writing line (Fig. 118) and when input is recognized inserts it into the text. 

MARK TEXT symbols - "<" and ">" A less than and greater than symbol drawn at the beginning and 
end of a block of text. The marked text is displayed in reverse video and then special block functions can 
10 be performed. 

DELETE MARKED TEXT - A delete symbol drawn within marked text will erase the marked text and 
reformat. 

MOVE MARKED TEXT - An insert symbol drawn anywhere within the text modes the marked text to the 
indicated position, deletes it from its original position and reformats the text. 
75 REPLACE MARKED TEXT - An insert symbol drawn within the marked text displays an input line and 
replaces the marked text with the inputted text. 

The Editing Symbols described above can be changed to the particular Editing Symbols preferred by 
each user, thereby customizing the Editor and preventing new users from having to learn unfamiliar Editing 
Symbols. 

20 Further modifications and enhancements to the present system will be apparent to those skilled in the 
art. For example, the common characteristics of each Font Symbol could be extracted and organized into a 
synthetic symbol. The synthetic symbols's characteristics could then be exaggerated to maximise their 
variance from all other synthetic symbols. This would create a very compact, optimal database. On the 
other hand, as an example, a database created by the described preferred embodiment of the invention 

25 usually results in two to three different characterizations for each symbol. 

The system has numerous useful applications, almost without limitation. The most obvious applications 
are text editing and filling out and modifying forms. Some of the many other applications that may not come 
to mind as readily are writing in languages utilizing large numbers of symbols like Japanese or Chinese, 
writing in Arabic and similar languages made up of a limited number of complex symbols; writing chemical 

30 equations, including those involving organic compounds; writing music (a "window" with five parallel lines 
can be provided for musical applications); writing symbols and codes for graphic manipulation of data, 
including the transfer of graphic data to a spreadsheet; in education, as where predetermined questions are 
presented on the screens and the answers written in long-hand; as in teaching mathematics, as when 
numbers are manually inserted in equations and the equation analyzed to determine the result using those 

35 numbers; in CAD/CAM applications involving symbols, geometric shapes and the like. 

We have described a particular preferred embodiment of a keyboard less computer on which usual 
computer functions are performed by writing in a normal manner with a pen-like stylus on an input screen 
placed directly over a flat display. The keyboardless computer illustrated is ideally configured for use by 
non-keyboard oriented individuals, by keyboard individuals for whom the utility of the computer is 

40 enhanced, and in various settings and applications in which keyboard entry is awkward or impossible. 
Computer-based information and application software can be loaded into a portable device for later viewing, 
manipulation of text and data, and adding new text and data in a normal handwriting mode. Thereafter the 
user may transmit this computer text to another computer, a similar device, an external electronic storage 
device, a hard copy printer, or through a telecommunications system. The computer is capable of 

45 recognising Handwritten Symbols with a high degree of accuracy and of "learning" individual styles of 
handwriting. The data and commands can be input with the aid of a stylus. 

Although the invention has been described in terms of a selected preferred embodiment encompassing 
the apparatus and methods aspects of a keyboardless computer system, the invention should not be 
deemed limited thereto, since other embodiments and modifications will readily occur to one skilled in the 

50 art. 
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APPENDIX I 

'h* (l)ns=lx= (-17..0..19) y = (-6. .18. .60) 

1) np = 20 x = (-17..0..19) y = (-6. .18. .60) 
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42 


-57, 


34 


-85, 


26 


-71, 


19 


-76, 


10 


-64, 


3 


-89, 


-3 


-72, 


-9 


-85, 


-17 


-72, 


-24 


47, 


-17 


56, 


-11 


43. 


-3 


39, 
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11, 


7 


_o 


5 


-57^ 


-3 


-71, 


-11 


-71, 


-18 



25 



30 



35 



40 



45 



50 



(1) ns = 2 x = (-27. .0. . 

1) np = 14 x = (-27, 

37 

4, 36 

-39, 32 

-4S, 25 

-58, 16 

-71, 8 

-72, 2 

-64, -4 

-89, -11 

-72, -17 

-80, -24 

-69, -30 

-80, -36 

2 ) np - 20 x = ( -21 . 

27, 43 

-80, 37 

-71, 29 

-98, 23 

-80, 16 

-76, 8 

-85, 1 

-89, -6 

-80, -12 

-60, -IS 

-64, -25 

-64, -31 

48, -25 

56, -18 

48, -12 

64, -6 

89, 1 

-124, 0 

-124, -2 



37) y = (-20. .18. .61) 

.-12. .-2) y = (-20. .22. .61) 



,10. .37) y = (-15. .16. .59) 
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5 1 REM 2 SERIES COPYRIGHT: LINUS TECHNOLOGIES INCORPORATED , RESTON , VA. , 7/30/85 

10 DEFINT A-Y : REM all integers except z 

20 DIM X( 2000 I , Yl 2000 1 , Dl 9 , 800 ), DN ( 300 ), DC ( 300 ) , SX ( 80),SY( 80),SL( 80), SCI 80) 
,SA( 80>,SXX( 80KSXNI 80),SYXl BOJ.SYNt 80 J ,SNO( 80 ) ,MTITL$ t 20 ) , A$ (1 500 ) ,LL { 24 ), 
AS$(200> ,SLET180I ,DBC(9, 10) .A5TRT180) , AENDI 8 0 ) 
25 PRINT FRE(O) 

1Q 30 PRINT m Z SERIES COPYRIGHT: LINUS TECHNOLOGIES INCORPORATED, RESTON, VA., 7/30 

/85" 

40 IMARGN=5 : IMARGX=6 2 : JTOP= 1 : JBTM=15 : JMENU=I60 :CR$=" " 

50 KYBD=0 : LHT=16 : SXTT= 4 : JTX=0 : ACPT=1000 

60 LHT4=LHT\4 :LHT3=LHT\3 :LHT5=LHT\5 :LHT2=LHT\2 

70 PRINT -PLEASE INPUT PEN SWITCH. . .-1 TO RERUN. 0 TO RUN , 1 TO RUN & STORE" : IN 
PUT SWTPN 

75 7 5 IF SWTPN > -1 THEN GOSUB 107 00 : REM TOUCH TECH SCREEN IN ASCII @ 9 600 

80 PRINT "PLEASE INPUT DEBUG ...-5 TO 5 ■ : INPUT DBUG 

85 PRINT -PLEASE INPUT ACCEPTANCE CRITERIA ... 1 0 TO 1000 * i INPUT ACPT 

90 PRINT -PLEASE INPUT EDIT/LEARN SWITCH... 1 TO EDIT": INPUT EDLN 

92 PRINT -PLEASE INPUT KEYBOARD SWITCH ... 0 FOR KEYS. 1 FOR MENU - : INPUT KYBD 

95 PRINT "PLEASE INPUT RESTART SWITCH... 1 TO RESTART WITH STORED DATA BASE- : INP 

UT RESTRT 

20 97 IFAVE=0 : IF SWTPN=-1 THEN IFAVE=0 : REM PRINT * PLEASE INPUT AVERAGING SWITCH. . 

.1 TO READ U AVERAGE, 2 TO AVERAGE INPUT** : INPUT IF AVE 
100 KEY OFF : SCREEN 2 :CLS 
120 IF RESTRT = 1 THEN GOSUB 1 2700 

130 IF EDLN = 1 THEN GOSUB 12100 : REM READ IN TEXT 
140 IF EDLN = 1 THEN GOSUB 600 : REM PRINT TEXT 

150 IF SWTPN = 1 THEN GOSUB 7200 

151 IF SWTPN = -1 THEN GOSUB 7 000 
170 IF EDLN <> 1 THEN GOSUB 6100 
180 IF EDLN <> 1 THEN GOSUB 12500 
20 0 REM EDIT 

205 LINY«0 

210 IF SWTPN = -1 THEN GOSUB 7100 ELSE GOSUB 2600 : REM TOUCH TECH SCREEN IN ASC 
30 II <? 9600 

211 IF SWTPN =1 THEN GOSUB 7300 
22 0 REM RECOGNIZE SYMBOLS 

221 IF NP > 2000 THEN LOCATE 23,1: PRINT -TOO MANY POINTS - : STOP 
22 2 IF NP < 4 THEN GOTO 200 

230 GOSUB 3000 :GOSUB 3400 : REM - , O " . | , ~ , \ . / . P , E 

240 ON NCHAR GOTO 1200,1300,1700,6200,6700,6500,6600,12500,6900,200 :REM DELETE, 
35 MOVE . FORMAT , INSERT , WRITE OVER , REMOVE, PARAGRAPH, INSERT CHARACTER , END 

250 GOTO 200 
500 END 

600 REM DISPLAY TEXT FROM SINGLE CHARACTER ARRAY 
605 LOCATE 23,20: PRINT • 

610 1=11 : J=JJ : REM INITIAL DISPLAY POINT FOR NCI 
« 620 FOR K«NC1 TO NC 

630 IF A$(K) <> CR$ THEN GOTO 660 

640 LOCATE J, I : PRINT - - : 1=1+1 : IF I < 81 GOTO 640 
650 J=J+t : I = IMARGN : LL(J)=K : GOTO 740 

660 IF K IMARGX THEN GOTO 670 ELSE IF A$(K)*=" * GOTO 690 
67 0 IF J> JBTM THEN LOCATE 21,20: PRINT "READY* , : RETURN 

680 LOCATE J, I : IF I = IN THEN GOTO 710 ELSE PRINT A$tK) : 1=1 + 1 : GOTO 740 
45 690 LOCATE J, I : PRINT - - : 1 = 1 + 1: IF K 81 GOTO 690 

700 J«J+1 : IoIMARGN : LL(J>=K+1 : GOTO 740 
710 IF A$(K> O - * THEN GOTO 730 
720 IF LL( J ) =K GOTO 740 
730 1=1+1 : PRINT A$(K) 

740 IF I> IMARGX+10 THEN GOTO 690 ELSE NEXT K 
7 50 IF J> JTX THEN GOTO 790 
60 7 60 JS=J 

765 LOCATE J, I : PRINT - 1 = 1+1: IF I< 81 GOTO 765 

770 J=J+1 : I = IMARGN : IF J > JTX THEN GOTO 780 ELSE GOTO 765 



55 



17 



EP 0 645 730 A1 



7 80 J-JS 

790 JTX^J : LOCATE 23,20: PRINT "READY", : RETURN 
1200 REM DELETE 

1 205 XX=SXXM ) : XN = SXNM): YX=SYXM»: YN = SYNIU 

1206 IF NSTRK=1 GOTO 12 20 

1210 FOR L=2 TO NSTRK 

1212 IF XX < SXXIL) THEN XX=SXX(L) 

1214 IF XN > SXN(L) THEN XN=SXN(L* 

1216 IF YX < SYX(L) THEN YX=SYX(L» 

1218 IF YN > SYNIL) THEN YN-SYN(L) 

1219 NEXT L 

7 0 1220 IN=IKARGN 

1225 I*INTIXX/8)+l : IF I < IN THEN I»IN 

1 230 J=INT<YX/8 J+1 : ML=LL( J ) :M2=ML+I-IN : IF A$(ML>«='* * THEN M2=M2+1 

1231 IF A$<ML)^CR$ THEN M2=M2+1 

1232 I=INT(XN/8)+1 : IF I < IN THEN" I=IN 

1234 J=XNT(YN/8 ) + l : ML=LL( J ) :M1=ML+I-IN : IF A$ ( ML ) - * " THEN M1=M1+1 
1240 IF A$(ML)-CR$ THEN M1=M1+1 
75 1250 MD=M2-M1+1: IF MD < 0 THEN MD=M1-M2+1 : Ml =M2 

1260 FOR M=M1 TO NC : AS ( M J =A$ I M+MD ) : IF M+MD=NC THEN GOTO 1270 ELSE NEXT M 
1270 FOR M=NC-MD+1 TO NC :A$<Mi= - - : NEXT M 
1 280 11 = 1: JJ=0: NCl*=M1 : GOSUB 600 
1290 NC-NC-MD : GOTO 200 

1300 REM MOVE INDICATED BY OPENING BRACKET 
1305 IN=IMARGN 
20 1310 XX=SXX( 1 ) : XN=SXN ( U : YX=SYXM ) : YN«SYN I 1 ) 

1320 XP=iXN+XX)/2: YP=(YN+YX)/2 :REM AVE BRACKET POSITION 

1 330 I = INTlXP/8 1 + 1 : J=INT(YP/8I+1 :REM CONVERT X,Y TO I ,J 

1 340 ML=LL(J): M1=ML+I-IN : REM CALC A ARRAY INDEX OF LETTER AT I, J 

1350 IF A$(ML)=* - THEN M1=M1+1 

1360 IF A$tMLJsCR$ THEN M1=M1+1 

1390 XX=SXX( 2 ) : XN=SXN 1 2 ) : YX=SYXI2): YN=SYN<2> 

14 00 XP=MXN+XXI/2: YP=(YN-YX)/2 : REM AVE BRACKET POSITION 

1410 I = INT(XP/8 > + l : J= INT ( YP / 8 I + 1 : REM CONVERT X.Y TO I.J 

1 420 ML=LLIJ): M2=ML+I-IN : REM CALC A ARRAY INDEX OF LETTER AT I, J 

1430 IF A$IML)=* " THEN M2==M2+1 

1440 IF A$IML)=CF$ THEN M2=M2+1 

14 50 M3=0:REM INITIATE COUNTER 

30 1460 FOR M=M1 TO M2 : REM LOOP OVER PORTION CUT 
1470 M3=M3+1 : REM INCREMENT COUNTER 
1480 ASS(M3 )«A$(M) :NEXT M : REM COPY CUT TEXT 
1490 XX=SXX(3): XN=SXN(3I: YX=SYXt3): YN=SYN<3> 

15 00 REM INSERT WITH M3 LETTERS STORED IN AS$ & xx,xn,yx,yn 
1 520 XP=(XN+XX)/2 : YP= { YN+YX J /2 : REM AVE BRACKET POSITION 
1530 I = INTtXP/8 : J=INT( YP/8 } + 1 : REM CONVERT X,Y TO I, J 

35 1 540 ML=LL t J ) : M3=ML+I-IMARGN : REM CALC A ARRAY INDEX OF LETTER AT I, J 
1550 IF A$ I ML ) = " " THEN M3=M3+1 
1560 IF A$ (ML) =CR$ THEN M3=M3+1 

15 70 MD=M2-M1+1 : IF MD > 0 THEN GOTO 1580 ELSE GOTO 200 

15 80 FOR M-Ml TO NC : REM LOOP OVER A$ ARRAY POSITIONS TO BE SHIFTED 
1590 A$(M)=A$(M+MD) : REM SHIFT A$ ARRAY LETTERS 

16 00 IF M+MD = NC THEN GOTO 1610 ELSE NEXT M : REM STOP AT END OF ARRAY =NC 

40 1610 IF M3 > M2 THEN M3=M3-MD : REM OFF SET INSERTION POINT TO ACCOUNT FOR DELETI 
ON 

1 6 20 FOR M-NC-MD TO M3 STEP -1 ; REM SPRED ARRAY TO MAKE ROOM FOR INSERTION 
16 30 AS t M+MD J «A$ t M ) : REM MOVE LETTERS 
1640 NEXT M 

1 6 50 FOR M-1 TO MD : REM LOOP OVER CHARACTERS 
45 1 660 A$(M3+M-U=AS$(M) : REM INSERT LETTERS 
T6 70 NEXT M 

1680 CLS: 11= IMARGN : JJ= JTOP : NC1=1 : GOSUB 600 

1690 GOTO 200 

1700 REM REFORMAT MARGINS 
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1720 XP=(SXN{L)+SXX(LI )\2 : REM AVE BRACKET POSITION 

1 730 I«=XP\8 + 1 : REM CONVERT X TO I 

1740 IF ABS( I-IMARGN ) <ABS ( I-IMARGX ) THEN IMARGN=I ELSE IMARGX=I 
1750 CLS: I 1= IMARGN : JJ=JTOP: NCl = l : GOSUB 600 
1 760 GOTO 200 

2600 REM TT ACQUIRE POINTS IN ASCII @ 9600 WITH TIME OUT 
2603 M=20 

2605 IF LOC(2! > 0 THEN IPUT$=INPUT$ 11 , 2 ) : GOTO 2605 

2606 PRINT K2,"G* :N = 0 

2610 IF LOC(2l < 40 THEN GOTO 2610 

2620 NU=0: INPUT 8 2 . XP . YP : INPUT *2,XP,YP : INPUT #2,XPA,YPA : INPUT #2,XP0,YPO 
2630 N=N+1 : XPA= I XPA+XPO ) \2 : YPA= I YPA+YPO ) \2 : XPO^XPA: YP0=YPA: X(N)=XPA: YIN) 
PA 

26 40 INPUT *2,XP,YP 

2653 XPA=(XP+3"XPA) \4 : YPA» ( YP+ 3 * YPA I \ 4 

26S7 IF ABStXPA-XPO I +ABS ( YPA-YPO ) < 3 THEN GOTO 2672 

2660 I« 3 2 * XPA\ AX+BX : J= 3 2 * YPA\ AY +BY : PSET I I , J) 

2670 N=N+ 1 : X ( N I =XPA : Y { N ) = YPA : XP0=XPA: YP0=YPA 

2 6 72 IF LOC(2> > 9 THEN GOTO 26 40 
2675 MM=0 

2680 MM«MM+ I : IF LOC(2> > 9 THEN GOTO 2640 ELSE IF MM < M THEN GOTO 2680 
2685 X(N)=-X(N); IF XIN) = 0 THEN X(N)=-1 

2690 IF LOC(2i > 39 THEN GOTO 2620 ELSE NU=NU+ 1 : IF NU < 4000 THEN GOTO 2690 

2695 PRINT 32, "S": NP=N 

2696 IF DEBUG <> 0 THEN LOCATE 20,2 :PRINT *NP=*,NP 

2697 IF IFAVE=2 THEN GOSUB 4 200 

2 6 98 IF DEBUG <> 0 THEN LOCATE 20,12 : PRINT NP 
269 9 RETURN 

3000 REM CALCULATE CHARACTERIZATION OF STROKES 
3005 REM GOSUB 4 200 : REM 2 POINT AVERAGE POINTS 

3010 A1=0: A2=0: DA1=0: DA2=0: SEND=0: ZX=0 : ZY=0: ZA=0 : ZDA=0: ZL=0: NSP=0 
3015 NSTRK=0: XX=ABS(X(1)|: XN=XX : YX=YH): YN=YX: Ml=l: M2=1: M3=2 : NLONG=0 

3 0 20 FOR M=2 TO NP 

3025 IF X(M» < 0 THEN SEND-2 :X(M)=-X(M> 
3030 IF M > Ml GOTO 3035 ELSE 3391 
3035 IF XtMl ) < 0 THEN M1=M: GOTO 3391 

3040 DX=X(MI-XIMU : DY=Y(M) - Y(M1 ] : ADX=ABSIDXJ: ADY-ABS ( DY ) : DS*=ADX+ADY 

3043 IF AY > 0 THEN DY=-DY 

3044 IF DS < 3 THEN GOTO 3391 

3052 IF X(M1 < XN THEN XN=X(M) 

3053 IF XIM) > XX THEN XX=X(M) 

3054 IF Y(MI < YN THEN YN«Y ( M ) 
3056 IF Y(MI > YX THEN YX^Y ( M > 

3058 ZX=ZX+X(M) -DS: ZY=ZY+Y I M ) "DS : ZL=ZL+DS: NSP=NSP+1 
3060 IF ADX < ADY THEN GOTO 3080 : REM ENCODE ANGLE 0-255 +-n256 
3070 IF DX > 0 THEN A1=64-(32*DY) \DX ELSE Al =1 92- { 32*DY ) \DX 
3075 GOTO 3065 

3080 IF DY > 0 THEN Al = I32*DX)\DY ELSE A1 =1 28 + t 32*DX) \DY 

3085 IF NSP>M2 THEN GOTO 3090 

3086 IF AKO THEN A1=A1+256 
308 7 ZA=ZA+A1*DS 

3089 GOTO 31 90 

3090 IF A1-A2 < -127 THEN A1-A1+256 : GOTO 3090 
3095 IF A1-A2 > 128 THEN AT*=A1-2S6 : GOTO 3095 
3100 ZA=ZA+A1*DS 

3110 DA1=A1-A2 
3115 ZDA=ZDA+DA1 

3120 IF NSP > M3 THEN ADA= ( DA 2 1- ADA I / 2 ELSE ADA=DA1 : ASTRT ( NSTRK+ 1 ) = ( A 1 +A2 ) \ 2 

3130 IF NSP ■ M3+1 THEN ASTRT (NSTRK+1 )=(A1+A2I\2 

3140 IF ABS ( DA1 -ADA ) > 80 THEN SEND=1 

3150 IF ABStDAH > 64 THEN SEND=1 

3190 IF SEND^O THEN GOTO 3 300 

3 2 00 REM END OF STROKE 
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3205 IF SEND = 1 THEN GOTO 3220 

3206 X(M)=-X(M) 

3208 SEND=0: SZL=ZL: SZL=32 *SZL\AY : IF SZL < 0 THEN SZL=-SZL 

3209 IF NSP < 3 THEN GOTO 3210 ELSE IF SZL > 2 THEN M1=M+1: GOTO 3250 

3210 IF NSTRK > 0 THEN IF X ( SNO I NSTRK I ) > 0 THEN X i SNO ( NSTRK I ) =-X i SNO I NSTRK > ) 
6 3212 M1=M+1 

3215 GOTO 3280 

32 20 SEND=0: SZL=ZL-DS: SZL=32*SZL\AY : IF SZL < 0 THEN SZL=-SZL 

3226 IF N5P < 4 THEN GOTO 330O ELSE IF SZL < LHT5 THEN GOTO 3300 

3230 Ml =M 

3240 ZX=ZX-XIM) *DS 

324 1 ZY=ZY-Y(M) *DS 

32 4 2 ZL^ZL-DS: ZA«ZA-A1*DS: ZDA=ZDA-DA1 
3 24 3 NSP=NSP-1 

3 2 50 NSTRK=NSTRK+T 

3253 SL(NSTRK)*50*SZL\LHT: AEND ( NSTRK I = ( A3+A4 J \ 2 : SC I NSTRK ) «ZDA : AZA=ZA/ZL 

3254 IF AZA > 255 THEN AZA=AZA-256 : GOTO 3254 : REM USE 128 HERE FOR DIRECTION I 
NDEPENDENCE 

75 3 256 IF AZA < 0 THEN AZA=AZA+256 : GOTO 3256 

3 2 60 SX( NSTRK )=ZX/ZL: SX i NSTRK I =3 2 * SX t NSTRK 1 \AX+BX 
3 26 2 SY(NSTRK)=ZY/ZL: SY ( NSTRK ) =32 *SY I NSTRK ) \AY+BY 
3264 SA ( NSTRK ) = AZA 

3270 SXX(NSTRK)=3 2*XX\AX+BX: SXN < NSTRK ) =3 2*XN\AX+BX : SNO(NSTRK)=M 
3272 IF AY < 0 THEN SYN ( NSTRK ) = 32 " YX\AY+BY : SYX ( NSTRK ) =3 2 *YN\AY+BY : GOTO 3 28 0 
3 27 3 SYXINSTRK)=3 2*YX\AY+BY : SYN ( NSTRK ) = 32 *YH\AY+BY 
20 3280 IF DBUG > 3 THEN LPRINT USING " * * t * * tt - ; M , NSP . X ( M ), Y ( M > . ZL, ZA , AT , DAI , ADA , ZDA 
, (A1+A2+A3+A4 )\4 ELSE IF DBUG < -3 THEN PRINT USING * # # #* t U - ; M, NSP , X { M ) , Y t M ) , ZL, 
ZA,A1 , DAI /ADA, ZDA. (A1+A2+A3+A4 )\4 

3 28 1 IF DBUG > 2 THEN LPRINT " SL , SA , SC . AS , AE " , SL ( NSTRK ) SA ( NSTRK ) SCt NSTRK) ASTR 
T( NSTRK) AEND i NSTRK) ELSE IF DBUG < ~2 THEN PRINT " SL , SA , SC , AS , AE " , SL ( NSTRK ) SA( 
NSTRK) SC ( NSTRK ) ASTRT (NSTRK) AEND (NSTRK) 

3285 A1=0: A2=0: DA1=0: DA2=0 : DS2=0 : DSl =0 : SEND=0 : ZX=0 : ZY=0: ZA = 0 : ZDA=0: ZL=0 
25 3290 XX=ABS(X(M1 ) ) : XN=XX :YX=Y(M1I: YN=YX : NLONG=0 :NSP=0 
3295 GOTO 3391 
3 300 REM FINISH LOOP 
3310 Ml =M 

3320 DA3=DA2 : DA2=DA1 : DS2=DSl : DSt=DS: A4=A3: A3=A2: A2=A1 

3390 IF DBUG > 3 THEN LPRINT USING t»" ; M, NSP . X (M ) , Y (M ), ZL , ZA , Al , DAK ADA , ZDA 
. (A1+A2+A3+A4 >\4 ELSE IF DBUG < -3 THEN PRINT USING •«#»*« ft m ;M, NSP ,X(M) .YIM) . ZL, 
ZA,A1 ,DA1 , ADA , ZDA , (A1+A2+A3+A4 )\4 

3391 NEXT M 

33 92 IF DBUG > 1 THEN LPRINT " INOX Y L A CXXXNY 
X YN ST ND" 

3393 IF DBUG < -1 THEN LOCATE 9,1 : PRINT " INOXYLAC 
XX XN YX YN ST ND" 
35 3394 IF DBUG < -1 THEN LOCATE 10,1 : FOR 1=1 TO NSTRK : PRINT USING "t#*ftt*;I, SNO ( I 
) , SX t I ) . SY ( I ) , SL ( I ) , SA II), SC 1 1 ) , SXX t I I , SXN ( I ) , SYX 1 1 > , SYN 1 1 ) ,ASTRT(I) , AEND ( I ) :NE 
XT I 

3395 IF DBUG > 1 THEN FOR 1 = 1 TO NSTRK: LPRINT USING ;I , SNO { I ) , SX ( I ) , S Y t 
I ) . SL ( I ) , SA ( I ) , SC 1 1 ) ,SXX< I ) .SXN! I ) ,SYX( I ) , SYN ( 1 ) .ASTRTt I) , AEND ( I ) :NEXT I 

3396 IF NSTRK > 1 THEN GOSUB 4 000 ELSE GOTO 3399 

3397 IF DBUG < -I THEN FOR 1=1 TO NSTRK: PRINT USING r I , SNO ( I ) , SX ( I ) , SY ( I ) 
40 , SL ( I ) * SA ( I > , SC ( I ) , SXX ( I > , SXN ( I ) , SYX ( I ) , SYN ( I ) , ASTRT ( I > # AEND i I ) : NEXT I 

3398 IF DBUG > 1 THEN FOR 1=1 TO NSTRK : LPRINT USING •*« *#* - ; I , SNO( I ) ,5X1 I ) ,SY( 
I ) , SL ( I ) ,SA( I ) ,SC(I i ,SXX< I) ,SXN( 1 ) .SYX 1 1 ) , SYN 1 1 ) , ASTRT (I ) , AEND ( I ) : NEXT I 

3 3 99 RETURN 

3 4 00 REM EDITING SYMBOL RECOGNITION 

34 10 FOR L=1 TO NSTRK 

3412 Al=(SA(L)+32)\64 : IF Al > 3 THEN A1=0 
45 34 16 IF Al O THEN IF SLID > 100 THEN NCHAR=3: RETURN: REM VERTICAL 

34 18 IF Al «= 1 THEN IF SL(L) > 60 THEN NCHAR=1 : RETURN: REM HORIZONTAL 
34 20 IF A1 « 2 THEN IF SLID > 100 THEN NCHAR«3: RETURN: REM VERTICAL 
3422 IF Al =3 THEN IF SL(L) > 60 THEN NCHAR= 1 : RETURN: REM HORIZONTAL 
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3430 NEXT L 

3500 REM CONVERT LETTER TO EDITTING CODE 
3510 ED=1 :GOSUB 3600 : ED =0 
3515 IF NLET > 0 THEN GOTO 3530 

3520 NCHAR«10: LOCATE 20,1: PRINT "TRY AGAIN*" : RETURN 
3530 IF CS$ » THEN NCHAR =1 : RETURN 

5 3532 IF NLET « 3 THEN IF CSS = THEN NCHAR=2 : RETURN 

3536 IF CSS = m ~ m THEN NCHAR= 4 : RETURN 

35 3 8 IF CSS = "\" THEN NCHAR =9 : RETURN 
3540 IF CSS = "P" THEN NCHAR= 7 : RETURN 
3542 IF CSS « ** E ** THEN NCHAR= 8 : RETURN 
354 4 IF CSS = THEN NCHAR<=6 : RETURN 

10 3 5 46 IF NLET= 1 THEN NCHAR=5 : RETURN 

3550 GOTO 3520 
3 590 RETURN 

3 600 REM PARCEL STROKES INTO CHARACTERS 

3602 IF DBUG > 4 THEN LPRINT USING ■■*««**# ■ ; SXX ( U , SXN { T ) , SYX M ) , SYN ( 1) , LHT4 

3603 SLET(1>=1: NLET= 1 : IF NSTRK= 1 THEN GOTO 3650 
3 6 05 FOR L=2 TO NSTRK 

75 3606 IF DBUG > 4 THEN LPRINT USING "Uttnttf; SXX ( L ) , SXN ( L ) , SYX I L ) , SYN ( L ) 

3 6 07 ICONT=0 

3610 FOR L1 = 1 TO 2 : IF L > 2 THEN GOTO 3615 ELSE IF L1=2 GOTO 3635 
3615 IF SXXIL-L1) > SXXtL) THEN GOTO 3630 ELSE IF I SXX ( L-Ll ) +LHT4 ) < SXN t L ) THEN 
GOTO 3635 

3620 IF ABSt ( SYX t L I +SYN t L ) )-(SYX(L-Ll )-*-SYN(L-Ll ) ) ) > LHT THEN GOTO 3635 
3629 IC0NT=L1 ."GOTO 3635 
20 3630 IF SXN(L-L1) > SXX(L)+LHT4 THEM GOTO 3635 ELSE GOTO 3620 

3635 NEXT LI 

36 4 0 IF ICONT=0 THEN NLET*NLET«- 1 : SLET t L ) =NLET : GOTO 36 4 9 

3645 NLET=SLET ( L-ICONT ) : SLET t L ) =NLET : SLET t L- 1 ) »NLET 

3646 IF DBUG > 3 THEN LPRINT **L , NLET , SLET * , L , NLET , SLET t L- 1 ) , SLET ( L- 2 > 
36 4 9 NEXT L 

36 5 0 REM LOOP OVER LETTERS 
25 3652 IF ED*1 THEN 2LINY = 0: FOR LMN= 1 TO NSTRK : ZLINY=ZLINY+SYX ( LMN ) +SYN ( LMN I : N 

EXT LMN : LINY=ZLIN Y/NSTRK : LINY= I 1 +LINY\ 1 6 ) *8 + 4 : IF DBUG > 0 THEN LPRINT "LINY= 
".LINY ELSE IF DBUG < 0 THEN LOCATE 22,1 : PRINT "LINY=" , LINY 
3655 MCKR=1 : NST=0 : XX=-1: XN= 1 000 : XXI =-1 00 : YX=- 1 :YN=1000 :YX1=-100 
3 6 56 FOR LSTRK= 1 TO NSTRK + 1 

3657 LST=LSTRK : IF LSTRK = NSTRK-*! THEN GOTO 3659 
30 3658 IF SLET ( LSTRK } = MCHR THEN NST=NST+): GOTO 3690 

3659 IF NST > 10 THEN LOCATE 23,1 : PRINT "TOO MANY STROKES", : STOP 

3660 FOR M=l TO NST : LMN^LST-NST+M- 1 

3661 DBCt 1 ,M)=SL(LMN) : DBC i 2 ,M) =SA ( LMN ) : DBC t 5 , M i =SC t LMN ) 

3662 DBC t 6 , M J = t 1 00 * ( SX ( LMN ) -SX I LST-NST ) ) ) \LHT : DBC I 3 ,M ) =ASTRT ( LMN ) :DBC( 4 ,M) =AEN 
D I LMN ) 

3 66 3 DBCt 7 , M ) = ( 1 00* ( SY( LMN ) -LINY ) ) \LHT : DBC(9,M)=( 50* t 3 2*Y ( SNOt LMN ) - 1> \AY+BY-L 
35 INY ) ) \LHT : IF LMN «1 THEN DBC(8,M)=t 50* ( 3 2* Y ( 2 1 \AY+BY-LINY J ) \LHT ELSE DBC(8,M)= 

( 50*t 32*Y(SNOlLMN-l ) +2 J \AY+BY-LINY ) ) \LHT 

3664 IF YX < SYX ( LMN ) THEN YX=SYX I LMN ) 

3665 IF YN > SYN(LMN) THEN YN=SYN I LMN ) 

3666 IF XX < SXX ( LMN ) THEN XX = SXX I LMN ) 

3667 IF XN > SXN ( LMN } THEN XN=SXN I LMN ) 

3668 IF DBUG > 0 THEN LPRINT USING - # * * * ft * ; MCHR , DBC t 1 , M ) , DBC ( 2 . M > , DBC ( 3 , M ) , DBC ( 4 
40 ,M) ,DBC(5,M) ,DBC(6,M) ,DBC( 7,M) , DBC t 8 , M I , DBC t 9 , M ) 

3669 IF DBUG < 0 THEN LOCATE 10+M.27: PRINT USING * * * t » 0 • ; MCHR , DBC (1 , M ) , DBC t 2 , M 1 
,DBC(3,M) ,DBCt 4 ,M) , DBCt 5 ,M) ,DBC< 6 ,M) , DBCt 7, Ml ,DBC( 8,M) , DBCt 9 , M ) 

3670 NEXT M :GOSUB 3700 

3671 NPOS='2+(XX+XN)\16-IMARGN'\2 

3672 REM PRINT "MCHR , NPOS , CHAR , ED" , MCHR , NPOS , CSS , ED :INPUT IJK 

3673 IF ED - 1 THEN NPOS-D : AS$ t MCHR i «=CS$ 

5 367 4 IF NPOS > 0 THEN IF NPOS < 4 0 THEN ASS t NPOS ) =CS$ 

3675 REM 

3676 IF ED <> 1 THEN LOCATE ( LINY-LHT+ 1 ) \8 , ( XX+XN ) \ 1 6+ 1 :PRINT CS$ 



50 



55 



21 



EP 0 645 730 A1 



20 



25 



36 7 7 REM 

3680 SXX(MCHRJ=XX : SXN(MCHRI=XN 
36S2 SYXIMCHRJ=YX : SYN I MCHR I =YN 

368B MCHR«=MCHR+1 : NST= 1 : XX1=XX: XX=-1 : XN=1000: YXl=YX: YX=-1 : YN=1000 
3690 NEXT LSTRK 
5 369 9 RETURN 

37 00 REM COMPARE TO DATABASE 

3710 Rl=16000 :M3=0 :IF NDBT=0 THEN GOTO 3900 

37 1 5 FOR M=1 TO NDBT 

3720 IF DNIM) <> NST THEN GOTO 37 9 0 : REM CHECK NUMBER OF STROKES 
3725 M2E=9 : IF LINY s 0 THEN M2E=6 
3730 R = 0 : FOR Ml =1 TO NST 
10 3735 FOR M2 = 1 TO M2E 

3738 DR«=ABS(D(M2,M3+M1 )-DBC(M2,M1 ) ) 

3740 IF M2 < 2 THEN GOTO 3744 ELSE IF M2 > 4 THEN GOTO 3744 

374 1 IF DR > 128 THEN DR=ABS I DR-25 6 ) : GOTO 374 1 : REM USE 64 FOR DIRECTION INDEPE 
NDENCE 

3742 IF ABS(D(5,M3+M1) ) > 20 THEN DR=2 0*DR\ IABS(D( 5 , M3+M1 ) I ) 
3744 IF DR > 300 GOTO 3790 
15 3 74 8 RssR+DR 

3749 IF R > Rl GOTO 3790 
3 750 NEXT M2 : NEXT Ml 
3780 MS1=M : R1=R 
3790 M3=M3+ DN (M ) : NEXT M 

38 00 REM IDENTIFY CHARACTER 

3803 CS$=CHR$( DCIMSl > ) . 

3804 REM 

3605 IF DBUG <> 0 THEN LOCATE 19,1 : PRINT R1,MSl,CS$ 
3806 REM 

3810 IF R1\NST > ACPT GOTO 3 9 00 : REM CHARACTER NOT FOUND 
3 8 30 RETURN 

3 900 REM LEARN NEW CHARACTER 
3902 IF LERN = 0 THEN CS$=CHR$ ( 2 ) : RETURN 

3904 IF KYBD-0 THEN LOCATE 23,1 : INPUT "CHARACTER NOT UNDERSTOOD , PLEASE TYPE IT 
IN: ",CS$ 

3906 IF KYBD = 0 THEN LOCATE 23,1 :PRINT " 

3910 IF KYBD= 1 THEN GOSUB 11400 

3912 IF KYBD =0 THEN GOSUB 9600 

30 3914 IF KYBD «1 THEN GOSUB 11600 : CS$=CHR$ ( MSEL ) 

3915 IF CS$^ W ~" THEN GOTO 3960 

3917 IF ED=1 THEN RETURN 

3920 IF NDBT = 300 THEN PRINT "Too Many Symbols": STOP 

3926 NDBT=NDBT+ 1 : DN ( NDBT I «NST : DC t NDBT ) =ASC ( CS$ ) 

3930 IF M3+NST > 800 THEN PRINT "Too Many Strokes" :STOP 

3936 FOR Ml =1 TO NST : FOR M2-1 TO 9 

35 3940 DIM2,M1+M3)=DBC(M2,MU : NEXT M2 : NEXT Ml 

3950 RETURN 

3960 CS$=* - 

3980 RETURN 

4 000 REM CONCATENATE STROKES 
4010 FOR L=2 TO NSTRK 

4014 IF XtSNOIL-U) < 0 THEN GOTO 4090 

4016 IF SLIL-U < 4 THEN GOTO 4 100 

4 0 20 ALNO=IABS(ABS(XlSNO(L) I ) -ABS I X t SNO ( L- T J ) ) ) *ABS < Y { SNOI L ) )-Y(SNO(L-l ) ) I ) 

4021 REM LOCATE 1,1: PRINT " L , ALNO " , L , ALNO 

4022 IF ALNO > 100 THEN GOTO 4090 : REM TOO FAR AWAY 
4024 ALNO= 8 * ALNO\ t SNO i L ) -SNO t L- 1 ) ) 

4 0 26 DD «ABS(ABS(X(SNO(L-1 ) I J -ABS I X t SNO ( L- 1 J + 2 1 ) ) + ABS t Y ( SNO I L- 1 ) )-Y(SNO(L-l ) + 2 ) I 

45 4027 REM PRINT "L . DD, ALNO" , L , DD , ALNO 

4028 IF DD > ALNO THEN GOTO 4090 : REM TOO FAR AWAY 

4030 IR1«=80*SC(L>\SL(L) : IR2 = 80*SC( L-1 ) \SL( L-1 ) 

4031 REM PRINT *L , IR 1 , IR2 * , L , IR 1 , IR2 
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4032 IF ABSIIRU > 350 THEN GOTO 4040 : REM SMALL WIGGLE 
4034 IF ABSIIRU < 80 THEN GOTO 4038 : REM STRAIGHT LINE 

4036 IF ABSIIR1-IR2) > ABS t IR1 +IR2 ) \ 3 THEN GOTO 4090 ELSE GOTO 4050 : REM TOO DI 
FFERENT CURVATURE 

4038 IF ABSIIR2) > 80 THEN GOTO 4036 ELSE IF IR1*IR2 < -2000 THEN GOTO 4090 : REM 
TOO DIFFERENT CURVATURE 

4039 GOTO 4050 

4040 IF ABSIIR2} < 350 THEN GOTO 4036 ELSE IF IR1*IR2 < 0 THEN GOTO 4090 
4 05 0 ASTND= ABS ( ASTRT ( L ) - AEND ( L- 1 > > 

4 051 REM PRINT " ASTND " , ASTND 

4052 IF ASTND > 128 THEN ASTND-ABSt ASTND- 256 ) : GOTO 4052 

4054 IF ASTND > 64 THEN GOTO 4090 ELSE IF ASTND > DD* ( ABS { IRU +ABS t IR2 ) ) \ 4 0 THEN 

GOTO 4 09 0 
4056 GOTO 4100 
4 090 NEXT L 
4 099 RETURN 

4 100 REM COMBINE STROKES 

4101 REM PRINT "COMBINING" , L , L- 1 

4110 L1=L-1 :SLL=SL(L1 )+SLlL) 

4 115 IF SLL *= 0 THEN LOCATE 23,1 : PRINT "TWO STROKES WITH NO LENGTH" :STOP 

4120 IF SXX(L) > SXXtLI ) THEN SXX I LI I =SXX I L ) 

4122 IF SXN(L) < SXNIL1) THEN SXN I LI ) =SXN I L ) 

4124 IF SYXiL) > SYXtL! ) THEN SYX I LI ) -SYX I L ) 

4 126 IF SYN(L) < SYNtLU THEN SYN I LI ) =SYN I L ) 

4 1 30 SXIL1 I=(SLIL1 I'SXIL! ) +SL I L ) *SX ( L I ) /SLL 

4133 SY1L1 )=(SL(L1 )*SY(L1 ) +SL I L ) *SY (L ) ) /SLL 

4 137 SAIL1 >«(SL<L1 J * S A i L 1 I +SLI L ) *SA J L ) )/SLL 

4 140 SLIL1 |=SLL +DD 

4150 SC(L1 1=SC(L1 >+SC(L)-M <SC(L1 )+SC(L> )*DD)\SLL 
4160 SNOIL1 )=SNOIL) : AEND ( LI J =AEND ( L ) 
4 170 NSTRK=NSTRK- 1 : IF NSTRK < L THEN RETURN 
4 180 FOR LL=L TO NSTRK 

4185 L1=LL+1 :SXXILL)=SXX(L1 ) : SXN t LL ) = SXN (LI) : SYX ( LL ) =SYX < LI ) : SYN ( LL 1 =SYN ( LI) 
4187 SX(LLJ=SX(LU : SY ( LL ) =SY ( LU : SA( LL ) =SA I LI ) : SC ( LL ) =SC (LI) : SNO I LL ) =SNO ( LI ) 
4 189 SL(LL)«SL(L1 ) : ASTP.T I LL ) « ASTRT ( L U : AEND(LL) =AEND( LI ) 
4190 NEXT LL 
4195 GOTO 4 010 

4 200 REM 2 POINT AVERAGE POINTS 
4210 M2=0 

4220 FOR M=2 TO NP STEP 2 :M1=M-l 
4230 IF XtMl ) > 0 GOTO 4240 

423S M2=M2+1 :X(M2)=X(MU :YIM2)*=YIMU :GOTO 4280 
4240 IF X(M) > 0 GOTO 4250 

4245 M2=M2+1 :X(M2J=X(M> :Y(M2)=YIM) :GOTO4280 

4250 IF ABStX(M)-X(M1 ) ) > 30 THEN GOTO 4280 

4260 IF ABS( Y(MI-Y(MU ) > 30 THEN GOTO 4280 

4270 M2-M2+1 : X t M2 ) « (X ( M ) +X t Ml )) \2 : Y (M2 > » ( Y I M ) +Y ( HI I i \2 

4 280 NEXT M 

4283 X(M2>«X(NP) 

4 2 85 NP«M2 

4 2 90 RETURN 

4 3 40 REM ORDER STROKES BY SXN 

434 2 XCH=0 :IF NSTRK < 2 THEN RETURN 

4345 FOR L*2 TO NSTRK 

4350 IF SXN1L) > SXN(L-I) THEN GOTO 4380 
4355 XCH=1 :L1=L-1 

4360 ST*=SXX(L1> : SXX ( LI) =SXX ( L ) :SXX(L)*=ST 

4361 ST=SXN(L1) : SXN ( L U =SXIJ ( L ) :SXNIL)*=ST 

4362 ST-SYNILU : SYN (LI > «= SYN ( L > :SYNIL)-ST 

4363 ST=SYX(L1) : SYX < LU <=SYX ( L J :SYXIL)=ST 

4364 ST=SX(L1 ) :SX(LU=SX(L> :SX(L)=ST 

4365 ST^SYILI ) : SY ( LU «=SY ( L ) :SY(L)=ST 

4366 ST»SA(LU : SA ( LI ) «=SA ( L ) :SA(L)=ST 
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4367 ST=SCtL1 ) : SCI LI ) =SC t L ) :SCILI=ST 

4368 ST=SLIL1> : SL < L 1 ) =SL ( L ) :SL(L1=ST 

4369 ST=SNOILl> : SNO< L1 ) =SNO ( L ) :SN01L)=ST 
4 380 NEXT L 

4390 IF XCH = 0 THEN RETURN ELSE XCH=0 :GOTO 4345 
5 6100 REM LEARN 

6105 LERN=1 :J=3: JJ=J : FOR 1=1 TO 40 :AS$tI)=" ' :NEXT I 
6110 LINY=8 * ( JJ+3 }- 1 

6120 LINE t 0 , LINY-LHT- 8 J - ( 6 3 9 , LINY+LHTX 2 ) , 0 . BF 
6130 LINE (8*IMARGN-1 ,LINY)-( 6 39 , LINY-LHT I , 1 ,B 
6140 LINE (607,LINY)-(639,LINY-LHTI , 1 , BF 

6150 IF SWTPN = -1 THEN GOSUB 7 100 ELSE GOSUB 8600 :REM TOUCH TECH SCREEN IN AS 
10 CII @ 9600 

6151 IF SWTPN = 1 THEN GOSUB 7 300 
6155 IF NP < 8 THEN GOTO 6 190 
6160 GOSUB 3000 

6170 GOSUB 3600 

61 BO LOCATE 1 8 , IMARGN : FOR 1 = 1 TO 40: PRINT ASS ( I ) ; : NEXT I 
75 6186 GOTO 6150 

6190 FOR 1=1. TO 4 

6192 IF AS$HJ - "G" THEN RETURN 

6194 NEXT I 

6195 GOTO 610S 

6 200 REM INSERT 

6201 XX=SXXM): XN=SXNM 1 ) : YX=SYX( U : YN=SYN(1) 
20 6202 XP=IXN+XXl/2 : YP=IYN+YXI/2 : REM AVE CHARACTER POSITION 

6 20 3 I^INT(XP/8 )+ I : J=INT t YP/8 ) + 1 :ISAV»I: JSAV=J: REM CONVERT X.Y TO I, J 

6204 ML=LL(J>: Ml =ML+I- IMARGN : REM CALC A ARRAY INDEX OF LETTER AT I, J 

6205 IF A$(ML1=" " THEN Ml-Ml+1 

6206 IF A$(ML)=CR$ THEN M1=MH1 
6 207 MSAV=M1 

6210 JJ=JSAV : FOR 1=1 TO 40 :AS$(I>=" " : NEXT I 
25 6215 LINY=*8MJJ+3)-1 

6220 LINE (0,LINY-LHT-B)-t639,LINY+LHT\2> , 0 , BF 
6230 LINE t 8 * IMARGN- 1 , LINY 1-1639 , LINY-LHT ) , 1 , B 
6240 LINE (607,LINY>-(639, LINY-LHT) , 1 , BF 

6250 IF SWTPN = -1 THEN GOSUB 7 100 ELSE GOSUB 8600 : REM TOUCH TECH SCREEN IN AS 
CII @ 9600 

6251 IF SWTPN = 1 THEN GOSUB 7300 
30 6255 IF NP < B THEN GOTO 6270 

6256 GOSUB 3000 

6257 GOSUB 3600 

6258 IF DBUG <> 0 THEN LOCATE 22, IMARGN: FOR 1=1 TO 40: PRINT ASS I I J ; : NEXT I 

6259 GOTO 6250 

6270 JB=0: J= 1 

6271 REM LOCATE 16,1: FOR 1=1 TO 40: PRINT A£$ (II;: NEXT I 

6272 IE=2 :FOR 1=40 TO 2 STEP -1 : IF AS$ ( I ) <> " " THEN XE-I+1 : GOTO 6274 

62 7 3 NEXT I 

6274 FOR 1=2 TO IE 

6275 IF ASSIII <> " * THEN GOTO 6280 
6 2 76 JB=JB+1 

6278 IF JB < 3 THEN GOTO 6288 
40 6280 J=J+1 : JB=0 

6282 AS$(J)=AS$(I) 

6288 NEXT I 

6289 REM LOCATE 17,1: FOR 1=1 TO 40: PRINT AS$ 111;: NEXT I 

6290 ASSt J+l ) = " " 

6295 IF J < 2 THEN IF ASS t 1 ) * " " THEN LOCATE 18,1 : PRINT *NO LETTERS, TRY AGAI 
N- :GOTO 6210 
45 6 3 00 MD=J+1 : M3=MSAV : I = ISAV : J=JSAV 

6310 REM LOCATE 20,1: FOR 1=1 TO 40: PRINT ASS t I ) ; : NEXT I 
6315 REM INPUT UK 
6320 GOTO 1620 
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6 500 REM REMOVE 

6510 XX-SXXM): XN=SXN(U: YX=SYXM ) : YN = SYNM) 

6 5 20 XP=(XN4XXI/2: YP= ( YN+ YX ) / 2 :REM AVE CHARACTER POSITION 

6530 I=INT(XP/8 »*1 : J=INTI YP/ 8 ) + 1 : REM CONVERT X,Y TO I, J 

5 6540 ML=LL( J) : Ml =ML+ 1- IMARGN : REM CALC A ARRAY INDEX OF LETTER AT I * J 

6550 IF A$ ( ML. ) - ** - THEN M1=M1+T 
6560 IF A$(MLi=CR$ THEN M!=MU1 
6570 M2=M1 
6590 GOTO 1 250 
6 600 REM PARAGRAPH 

10 6690 GOTO 200 

6 7 00 REM REPLACE A CHARACTER WHEN WRITTEN OVER 

6710 XX=SXX( 1 ) : XN=SXN(1): YX=SYX( 1 ) : YN-SYN (1 ) 

6 720 XP=*(XN+XX)/2: YPMYN+YXJ/2 : REM AVE CHARACTER POSITION 

6730 I=INT(XP/8 )+1 : J=INTt YP/8 ) + 1 : REM CONVERT X,Y TO I , J 

6740 ML-LL(J): M 1 =ML+ I- IMARGN : REM CALC A ARRAY INDEX OF LETTER AT I, J 
75 6750 IF A$(ML)=- " THEN Ml =M1 +1 

6760 IF A$<ML)=CR$ THEN M1=M1+1 
6770 A$(M1 )=AS$( t ) : LOCATE J, I :PRINT AS$ M ) 
6790 GOTO 200 
6 900 REM INSERT A LETTER 

6910 XX=SXXll>: XN=SXN ( H : YX=SYXM>: YN=SYNM) 
6920 XP=(XN+XX)/2: YP=MYN+YX)/2 : REM AVE CHARACTER POSITION 
20 6 9 3 0 I = INT(XP/8 i+1 : J=INT< YP/8 > + 1 : REM CONVERT X,Y TO I.J 

6940 ML=LL( J) : Ml =ML+ I -IMARGN : REM CALC A ARRAY INDEX OF LETTER AT I, J 

6 95 0 IF A$ i ML ) = " " THEN M1=M1-+1 
6960 IF A$(ML)=CR$ THEN M1=M1+1 
6970 ASSi 2)=AS$( t ) 

6980 M3=M1 :MD=1 
25 6 9 9 0 GOTO 1 570 

7 00 0 REM SET UP RERUN 

7 010 OPEN "I" , *1 , "POINTS. DAT" 
7020 INPUT #1 ,AX,BX,AY,BY 
7 03 0 RETURN 
7100 REM READ POINTS 
30 7110 INPUT *1 ,NP 

7120 FOR N=1 TO NP 
7130 INPUT #1 ,X(N) ,Y(N) 
7132 NEXT N 

7 133 IF IFAVE = 1 THEN GOSUB 4200 

7134 FOR N=1 TO NP 

7135 IF XIN) > 0 THEN 1= 32 *X ( N ) /AX+BX ELSE I = -3 2*X( N ) /AX+BX 

7136 J=32*YtN)/AY+BY 

7137 PSETII.J) 
7140 NEXT N 

7148 IF DBUG > 5 THEN FOR N=l TO NP : LPRINT N , X ( N ) ,Y(N) : NEXT N 

7149 IF DBUG <> 0 THEN LOCATE 20,2 : PRINT P NP=",NP 

7150 RETURN 

40 7 200 REM SET UP STORE POINTS 

7210 OPEN *0" , #1 , "POINTS.DAT" 

7220 PRINT f 1 r AX , BX , AY , BY 

7 230 RETURN 

7 3 00 REM STORE POINTS 

7310 PRINT *1 ,NP 
45 7 3 20 FOR N=l TO NP 

7330 PRINT *1, USING XtNJ.YIN] 

7340 NEXT N 

7 350 RETURN 

8600 REM TT ACQUIRE POINTS IN ASCII @ 9600 WITH INSERT 
8603 M=20 

50 8 6 0 5 IF LOCI 2) > 0 THEN IPUT$ = INPUT$ ( 1 , 2 ) : GOTO 8605 

8606 PRINT 12 , "G" :N=0 
86 10 IF LOC(2) < 4 0 THEN GOTO 8610 
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8620 NU=0: INPUT ff2,XP,YP : INPUT t 2 . XP , YP : INPUT *2,XPA,YPA : INPUT #2,XP0,YP0 
8630 N=N+1 : XPA= ( XPA+XPO ) \ 2 : YPA= t YPA+YPO ) \ 2 : XPO=XPA: YPO«YPA: X(N)*XPA: Y ( N ) «= 
PA 

8635 IF 32*XPA\AX+BX > 607 THEN GOTO 8695 
8640 INPUT #2,XP,YP 

8653 XPA=(XP+3*XPA)\4 : YPA= < YP+3 - YPA ) \ 4 

8657 IF ABS<XPA-X?0)+ABS( YPA-YPO) < 3 THEN GOTO 8672 

8660 1 = 32 *XPA\AX+BX : J= 3 2 * YPA\ AY+BY : PSET ( I , J ) 

8665 IF I > 607 THEN GOTO 8695 

8670 N=N+1 : X(N )=XPA :YIN)=YPA: XP0=XPA: YPO=YPA 
8672 IF LOC(2) > 9 THEN GOTO 8640 
8675 MM=0 

8680 MM=MM+1 : IF LOC I 2 ) > 9 THEN GOTO 8640 ELSE IF MM < M THEN GOTO 8680 
8685 XIN)e-X(N): IF X(N) = 0 THEN X(N>*-1 

8690 IF LOC(2) > 39 THEN GOTO 8620 ELSE NU=NU+1 : IF NU < 25000 THEN GOTO 8690 
8695 PRINT *2, B S": NP«=N 

8 696 IF DEBUG <> 0 THEN LOCATE 2 0,2 : PRINT *NP«",NP 

8697 IF IFAVE = 2 THEN GOSUB 4200 

8698 IF DEBUG <> 0 THEN LOCATE 20,12 :PRINT NP 
8 699 RETURN 

9600 REM TT ACQUIRE POINTS IN ASCII @ 96 00 WITH MENU 
9603 M=20 

9605 IF LOC(2) > 0 THEN IPUT$= INPUTS (1,2) : GOTO 9605 

9606 PRINT « 2 , "G" 
9610 11=0 :JJ=0 :N=0 
9640 INPUT »2,XP,YP 

9660 II=32*XP\AX+BX+II : JJ=32 * YP VAY+BY+ JJ : N=N+1 
9670 IF N < 10 THEN GOTO 9674 

9672 I=II\10: J=JJ\10: N=0 ;II=G: JJ=0 

9673 LINE t I + 1 , J+ 1 ) - ( I - I . J- U . 1 . B 

9674 IF LOC(2) > 9 GOTO 9640 

9675 MM=0 

96B0 MM=MM+1 : IF LOC ( 2 ) > 9 THEN GOTO 9640 ELSE IF MM < M THEN GOTO 9680 
9690 IF LOCI 2) > 9 THEN GOTO 96 10 ELSE NU-NU+1 : IF NU < 1500 THEN GOTO 9690 
9695 PRINT »2, "S" 
96 9 9 RETURN 

10700 REM SET UP TOUCH SCREEN FOR ASCII @ 9600 
10710 OPEN " I " , $ 2 , " CAL . DAT " 
10720 INPUT #2, AX, BX, AY, BY 
10730 CLOSE #2 

10740 OPEN "COM1 : 96 00 , N , 8 , 1 , CSO , DSO " AS *2 

10750 PRINT «2, "S" 

107 60 RETURN 

1 1000 REM UTILITIES 

11100 REM input selection menus 

11101 REM mnum= no. of items 

11102 REM mtitl$=titles 

11103 REM mi 0 ,mie ,mj0 ,mje=start & end points for boxes 
11105 IF KYBD= 1 GOTO 11400 

11110 LINE(MIO,MJ0 >-(MlE,MJE) , O , BF : LINE ( MIO , MJO I - ( MIE , MJE ) . 1 ,B 

11115 XX^MIO : MRATO=(MIE-MI0)\MNUM 

11120 FOR K-1 TO MNUM : LINE t XX, MJO )- (XX, MJE ) 

11130 MJA«=(MJO+MJE+8)\16 + 1 : LOCATE MJA, (XX + 20)\8 : PRINT MTITL$ ( K ) 
11140 XX=XX+MRATO : NEXT K 
11160 RETURN 

11300 REM DETERMINE MENU ITEM SELECTED 
11310 IF KYBD= 1 GOTO 11600 

1 1320 XX*=0 : FOR K= 1 TO MNUM : XX=XX+MRATO : IF I < XX THEN GOTO 11340 
1 1330 NEXT K 
113 4 0 MSEL=K 
1 1350 RETURN 

11400 REM DISPLAY KEYBOARD 
1 1405 JMENU=163 
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10 



15 



20 



25 
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11410 
11440 
1 1 450 
11460 
11470 
11460 
1 1 462 
11484 
11490 
1 1 600 
1 1 620 
11640 
1 1650 
1 1660 
1 1670 
1 1680 
1 1 690 
1 1 700 
11710 
1 1720 
1 1 730 
1 1740 
1 1 750 
1 1 760 
11770 
1 1 7B0 
1 1 790 
1 1 800 
11810 
11820 
1 1 830 
1 1 8S0 
1 2000 
12 100 
12110 
121 20 
121 30 
12140 
12143 
121 45 
121 50 
12155 
12160 
12170 
12175 
12180 
1 2500 
1 2505 
12510 
1 2520 
1 2530 
1 2540 
1 2550 
12560 
1 2570 
12580 
1 2590 
1 2700 
12710 
1 2720 
1 2730 
1 2740 
1 2750 



LINE f 1,160)-(639,199),0,BF 



FOR 1 = 



1 TO 



FOR 1= 1 TO 
FOR 1= 1 TO 
LOCATE 2 2,74 
LOCATE 2 4,54 



26 : LOCATE 2 2.2*1 : 
26 : LOCATE 2 4.2*1 : 
10 : LOCATE 22,2*1+52 
: PRINT -.*?". 
: PRINT -\ / • + - ( 



PRINT CHR5 ( 1 + 6 4 ) . 
PRINT CHR$( 1+96 ) , 
: PRINT CHR$( 1+47 ) 



FOR J=163 TO 195 STEP 16 
FOR 1= 3 TO 6 35 STEP 16 
LOCATE 1 , 1 



},;:"; CHR$ (34);" 
LINE( 3,J)-(629,J) :NEXT J 
LINE (I,163)-ll,195) : NEXT I 



NEXT I 
NEXT I 
: NEXT I 

■ $ 



: RETURN 

REM keyboard select from menu 

MSEL-(I\e+l )\2 

IF J > 179 THEN GOTO 11700 

IF MSEL < 27 THEN MSEL=MSEL+6 4 : 

IF MSEL < 37 THEN MSEL-MSEL+2 1 : 

IF MSEL=37 THEN MSEL«4 6 : RETURN 

IF MSEL=38 THEN MSEL=94 : RETURN 

MSEL=6 3 : RETURN 



IF 


MSEL 


< 


27 THEN MSEL=MSEL+96 : 


IF 


MSEL= 


27 


THEN 


MSEL= 


92 


: RETURN 


IF 


MSEL= 


26 


THEN 


MSEL= 


47 


: RETURN 


IF 


MSEL= 


29 


THEN 


MSEL= 


42 


: RETURN 


IF 


MSEL= 


30 


THEN 


MSEL= 


43 


: RETURN 


IF 


MSEL= 


31 


THEN 


MSEL = 


45 


: RETURN 


IF 


MSEL= 


32 


THEN 


MSEL= 


40 


: RETURN 


IF 


MSEL= 


33 


THEN 


MSEL= 


41 


: RETURN 


IF 


MSEL= 


34 


THEN 


MSEL = 


44 


: RETURN 


IF 


MSEL= 


35 


THEN 


MSEL= 


59 


: RETURN 


IF 


MSEL= 


36 


THEN 


MSEL= 


58 


: RETURN 


IF 


MSEL= 


37 


THEN 


MSEL= 


34 


: RETURN 


IF 


MSEL« 


38 


THEN 


MSEL- 


36 


: RETURN 



RETURN 
RETURN 



RETURN 



MSEL=3 8 : RETURN 
RETURN 

REM input output 

REM read in text 

OPEN "I" i II, "TEXT" : NC=0 

IF EOF t 1 ) THEN CLOSE ftl : GOTO 12160 

INPUT ft 1 , Al $ :KK=0 

FOR K= 1 TO LEN ( A 1 $ ) : KK=KK+ 1 : A$ ( NC+KK ) s=MID$ ( Al $ , K , 1 I 
IF LEN (A$t NC+KK ) )=0 THEN KK=KK- 1 
NEXT K 

NC=NC+KK:IF NC < 1500 THEN GOTO 12120 
PRINT "TOO MUCH TEXT" : STOP 
REM TEXT EDITOR INITIALIZATION 

1 1= IMARGN : JJ=JTOP : NC1=1 : REM DISPLAY ALL TEXT 

CLOSE in 

RETURN 

REM SAVE SYMBOL DATA BASE 

IF SWTPN <> 0 THEN CLOSE «1 

OPEN *0" , #1 , "SYM , DAT* 

PRINT #1 , NDBT 

M2=0 :FOR M=1 TO NDBT 

PRINT ftl , DNCM) DC(M) 

FOR M1=M2+1 TO M2+DN ( M ) 

PRINT #1, DM, Ml) D12.MU D(3,M1) DM, Ml) D(5,M1) D(6,M1) D(7,M1) 
NEXT Ml :M2=M2+DN<M) : NEXT M 
CLOSE ftl 
END 

REM RESTART READ IN DATA BASE 

OPEN "I" , If 1 , "SYM.DAT" 

INPUT ftl , NDBT 

M2=0 :FOR M= 1 TO NDBT 

INPUT ftl , DN I M ) , DC ( M ) 

FOR M1«M2+1 TO M2+ DNtM) 
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1 27 60 INPUT •l,DI1 ( M1),DI2,Mn ( D(3,MU,D14 ( M1),DI5,Ml),DI6,M1KD(7,M1) 
127 70 NEXT Ml :M2=M2+DN(M> : NEXT M 
12780 CLOSE «1 
127 90 RETURN 

30000 FOR 1^1 TO NSTRK : PRINT I , SNO ( I ) , SX ( I J , SY i 1 1 , SL ( I ) , SA ( I J , SC ( I J : NEXT I 
30010 FOR 1=1 TO NP:PRINT I,X(I),Y(I): NEXT I 

30020 FOR 1=1 TO NSTRK : PRINT USING * ft * * ft # * ; I , SNO t I ) , SX 1 1 ) , SY ( I ) , SL( I ) , SA( I ) , SC ( I 
J :N£XT I 

30030 FOR M1 = 1 TO NST : FOR M2 = 1 TO 5 :PRINT Ml , M2 , D ( M2 , Ml ) , DBC (M2 , Ml \ : NEXT M2 :N 
EXT Ml 
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APPENDIX III 



10 



15 



Al - angle of slope between Point M and Ml 

A2 - previous angle 

DAI - change in angle (A1-A2) 

SEND - flag for end of Stroke 

ZX - double precision X position summation by arc length weighting 

ZY - double precision Y position summation by arc length weighting 

ZA - double precision angle summation by arc length weighting 

ZDA - double precision change in angle summation 

ZL - double precision length 

NSP - number of Points in Stroke, counter 

NSTRK - number of Strokes 



XX - 


maximum X 










XN - 


minimum X 










YX - 


maximum Y 










YN - 


minimum Y 










Kl - 


Pointer to previous Point 










M2 - 


(1) minimum number of Points 
2t. angle 


for 


which 


to 


adjust for 


K3 - 


(2) minimum number of Points 
in change in angle 


for 


which 


to 


calculate change 



20 NLOKG - counter (not used) 

K - Pointer to Point being considered 

NP - number of Points 

X - array of X coordinate 

Y - array of Y coordinate 

DX - chance in X 
25 DY - change in Y 

ADX - absolute value of DX 

ADY - absolute value of DY 

DS - pseuco arc length ADX + ADY 

AY - calibration multiplier for vertical (Y) direction, e.g. 
AY * O mean coordinate system true downward 
30 ADA - running average change in angle 

DA2 - previous change in angle 
ASTRT - array for starting angle of Stroke 
SZL - single precision Stroke length 

SNO - array of number of the 1st Point for each Stroke 
LHT5 - height between lines divided by 5 in display coordinates 
35 used es a measure 

SL - array of Stroke lengths sealed to height between lines 
AEND - array of angles at end of Strokes 
SC - array of angle changes for eachs troke 
AZA - average angle through Stroke 

SX - array of averages centroid X coordinate for each Stroke 
40 SY' - array of averages centroid Y coordinate for each Stroke 

SA - array of averages angle (slope) coordinate for each Stroke 
SXX - array of maximum X coordinate for each Stroke 
SXN - array cf minimum X coordinate for each Stroke 
SYN - array of minimum Y coordinate for each Stroke 
SYX - array of maximum Y coordinate for each Stroke 
45 D5UG - debugging flag for printing 

INPUT - string of input bytes to be rejected 
XP, YP - input coordinates 
XPA - average 2 Points in X direction 
YPA - average 2 Points in Y direction 
XPO - initial X average on pan down 
50 YPO - initial Y average on pan down 

I - display pixel corresponding to X coordinate input 



J - display pixel corresponding to Y coordinate input 
55 N - counter of input Points 

MM - counter for timing pan up 
KU - counter for timing pan up 
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In addition to the foregoing it will be apparent that aspects of the invention may also be embodied in the 
following embodiments, including first an 

input/output or I/O device for providing input signals to a utilization device that are representative of a 
graphical character produced by a user and for receiving output signals from the utilization device and 
5 providing a visual representation of the output signals, said I/O device comprising: 

(a) a display means for providing a visual display of a graphical character in response to output signals 
provided by the utilization device; 

(b) an input screen means for producing a train of input signals as a graphical character or symbol is 
sketched by a user, said input screen means comprising a translucent input screen having top and 

10 bottom surfaces, said bottom surface being disposed with respect to said display surface such that the 
visual display produced by said display means is visible from said top surface of said input screen; and 

(c) means for determining the identity of said graphical character and providing output signals repre- 
sentative thereof to said display means. 

Preferably, in such an I/O device said display means comprises a display surface on which the 
75 graphical character is produced; and 

said input screen means bottom surface is disposed in close proximity above said display surface such 
that the visual display produced by said display means is visible from the top side of said input screen. 

Also preferably, in such an I/O device said input screen means includes a substantially flat plate, and 
wherein said input screen top surface is the top surface of said plate. 
20 Advantageously, such an I/O device may further include an indicating means operated by a user for 
cooperating with said input screen top surface such that said input screen means generates said train of 
input signals when said indicating means is in a predetermined position with respect to said plate top 
surface, and/or said input screen means may generate an input signal when said indicating means is in 
contact with said input screen top surface. 
25 In addition, in this I/O device 

(a) said plate may have electrical characteristics such that the voltage at any Point thereon varies in a 
known manner with the distance from the location at which a voltage is applied thereto; 

(b) said input screen means may further include means for selectively sequentially applying a voltage 
along one Cartesian axis of said plate and then along the other Cartesian axis of said plate: and 

30 (c) said indicating means may be elongate and have a tip at one end thereof and include a voltage 
detecting means that provides a signal representative of the voltage present where said tip contacts said 
input screen plate. 

The I/O device of this embodiment may also comprise the features that; 

(a) said plate has electrical characteristics such that the voltage at any Point thereon varies in a known 
35 manner with the distance from the location at which a voltage is applied thereto; 

(b) said input screen means further includes means for selectively sequentially sampling at an origin 
thereon the voltage along one Cartesian axis of said plate and then along the other Cartesian axis of said 
plate; and 

(c) said indicating means is elongate with a tip at one end thereof and includes a means for applying a 
40 known voltage to said tip; said sampling means detecting the voltages at said origin when said tip 

contacts said input screen plate and provides a signal representative of the detected voltages. 
In the I/O device of this embodiment said display screen preferably has a display area on which graphic 
symbols can be generated and a key input area by which command signals for the utilization device can be 
generated when contact is made with the corresponding area thereabove on said input screen. 
45 In a further aspect, the invention may also be embodied by a handwriting character recognition 
apparatus comprising: 

(a) a display means having a display surface for providing a visual display of graphical characters in 
response to a plurality of display signals; 

(b) a writing surface means for receiving graphical characters handwritten by a user, said writing surface 
50 means being disposed in close proximity with respect to said display means and comprised of a 

substantially translucent surface which has positional coordinates associated therewith; 

(c) an indicating means operated by a user in connection with said writing surface means for producing 
writing signals in cooperation with said writing surface means; 

(d) position detecting means for converting said writing signals into locating signals representative of the 
55 positional coordinates of said indicating means with respect to said writing surface; 

(e) a processing means for receiving and storing said locating signals as they are produced and for 
generating said display signals such that said display means can graphically display the graphical 
character after it has been produced by said indicating means; said processing means being comprised 
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of a character recognition means for comparing said produced locating signals representative of a 
graphical character with a plurality of stored database signals for identifying said graphical character. 
Preferably, in the handwriting character recognition apparatus said writing surface means includes a 
substantially flat plate which produces a signal when said indicating means is in contact therewith. 
5 The embodiment may include means for converting said positional coordinates into relational coordi- 
nates, and may also include means for dividing up a stream of locating signals into sections representative 
of a Stroke; and means for calculating Stroke characteristics which can be compared with Stroke 
characteristics of the database signals. 

The processing means of the handwriting character recognition apparatus of this embodiment prefer- 
w ably includes means for updating said stored database signals with said produced locating signals when no 
correct graphical character is identified. 

A method according to the invention may be embodied by a method of recognizing a Handwritten 
Symbol comprising the steps of: 

(a) generating a stream of locating signals for Points by moving a stylus about locations proximate to a 
75 writing surface means that generates a signal representative of said location of said stylus; 

(b) dividing up a stream of locating signals into sections representative of a Stroke; 

(c) calculating Stroke characteristics; 

(d) comparing said calculated Stroke characteristics with Stroke characteristics previously stored in a 
database; and 

20 (e) determining the best comparison and indicating if said best comparison is good enough to be a 
match. 

A further embodiment of the invention provides an apparatus for recognizing Handwritten Symbols and 
displaying Handwritten and Font Symbols on a screen comprising: 

(a) A visual display screen having the graphic capability to display Font Symbols and execute 
25 predetermined commands; 

(b) Hand-held means to write or draw Handwritten Symbols on or over said screen; 

(c) Means to display a true representation of the Handwritten Symbols on said screen as they are 
created; 

(d) Digitizing means to sense the position of said hand-held means and to convert the same into a series 
30 of electrical signals defining the position, size and shape of each Handwritten Symbol; 

(e) Means to compare predetermined characteristics of each digitized Handwritten Symbol with a 
database of predetermined characteristics of Font Symbols to assess the identity of each Font Symbol; 

(f) Means to convert the Handwritten Symbol to a predetermined Font Symbol or a command and to 
display the Font Symbol on the screen in close proximity to the screen area in which the Handwritten 

35 Symbol was originally entered or to execute the command, as the case may be 

This embodiment preferably includes a means to display predetermined fonts or symbols in the form of 
text or forms on said screen, whereby the Handwritten Symbols can be used to input information to 
complete a form or to edit a predetermined text; and may advantageously be implemented in a portable 
unit having dimensions up to about 400mm by 400mm by 100mm, a weight of up to about 7 kilo-grams and 

40 is adapted to include a self-contained power supply. 

The apparatus of the embodiment preferably further includes a means to display predetermined 
symbols in a font and command-representing symbols on said screen, to permit handwritten symbols 
representing each predetermined symbol to be manually defined on the screen and means to identify the 
manually defined symbol with the corresponding predetermined symbol in said database. 

45 Preferably, the apparatus of the embodiment also includes a means to modify said database when a 
Handwritten Symbol cases an erroneous Font Symbol to be displayed on the screen or conmand to be 
performed, as the case may be and/or may include a means to create a window of empty space on said 
screen upon a predetermined command for entering Handwritten Symbols in said window. 

In the apparatus of the embodiment, areas on said screen sensitive to touch to cause predetermined 

50 functions to be performed may advantageously be provided, and/or the screen may be substantially flat and 
adapted to be used in a substantially horizontal position. 

The apparatus of the embodiment may also advantageously include Softkeys on said screen and 
means to execute operational commands in response to touching any of said Softkeys. 
A further method aspect of the invention may be embodied in a microprocessor-based process for 

55 recognition, translation and display of Handwritten Symbols and execution of commands comprising the 
steps of: 

(a) Creating a personalized database for each individual user of the process by having the user initially 
input a Handwritten Symbol for each character corresponding with a font to be displayed or command to 
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be executed; 

(b) Determining a unite set of characteristics to characterize each such symbol and storing it in the 
database; 

(c) Writing or drawing Handwritten Symbols on a computer screen with a stylus; 

5 (d) Digitizing each Handwritten Symbol to identify the x,y coordinates of a multiplicity of Points defining 
the Symbol; 

(e) Processing the digitized characteristics of each Handwritten Symbol to determine the predetermined 
characteristics of the Symbol; 

(f) Search the database to find a "hit," a perfect or the closest correspondence to the characteristics; 

10 (g) Displaying the Font Symbol or performing the command most closely associated with the "hit" 
characteristics. 

This process may advantageously include the additional steps of: 

(a) Determining if the Font Symbol displayed or command performed is an error; and 

(b) If there is an error, re-inputting a Handwritten Symbol for the desired Font Symbol or command to 
75 modify the database, and/or may include the additional step of: 

(a) Substantially contemporaneously displaying the Handwritten Symbol on the screen as it is being 
written or drawn. 

In the process of this embodiment steps (b) and (c) preferably may involve a determination of the (i) 
length, (ii) average slope, (iii) centroid height above base line, (iv) curvature (rate of change in slope) and (v) 
20 comparison of the location of the centroid of each Stroke with the centroid of the Handwritten Symbol. 
Optionally, in the process, step (f) comprises: 

(a) Determining the value of each of the five characteristics; 

(b) Subtracting the database value of the characteristic from the newly-measured value; 

(c) Scaling the absolute values of each measurement to make each of the five measurements reasonably 
25 equivalent to the others; 

(d) Adding all five values; 

(e) If a predetermined threshold is exceeded, discarding the comparison and creating an error message; 
and 

(f) Searching the database to find a numerical "hit," or, if not feasible, the closest numerical match 
30 between the database value and the newly inputted value. 

In the process of the embodiment, the error rate is advantageously 5% or less, and/or the program for 
step (f) requires less than 6K of machine memory. 

Optionally, in the process embodying the invention said Handwritten Symbols include Editing Symbols 
and said process includes recognition of the Editing Symbols drawn on the screen and executing the 
35 Editing Functions represented. 

Further, in this process, if text in Font Symbol format is displayed on the screen, the process preferably 
may include the steps of: 

(a) Creating a window on the screen proximate to, but not overlying, an area of said text to be edited; 

(b) Inputting and displaying said Handwritten Symbols in said window; 

40 (c) Displaying the Font Symbols corresponding to said Handwritten Symbols in close proximity to said 
Handwritten Symbols. 

Also in the process embodying the invention, the following steps may advantageously be included: 
(a) Creating one or more Softkeys on said screen to cause the computer to perform operational 
functions; and 

45 (b) Touching one or more of said Softkeys to execute the corresponding functions. 

In addition, this process may include the steps of determining the x,y coordinates of from about 4 to 8 
Points per mm and from about 100 to 200 points per second in characterizing said Handwritten Symbols. 

Claims 

50 

1. A method for entering a handwritten symbol into a computer system, including the steps of; 

creating, in response to a predetermined command, a window area in at least a portion of a display 
area of a display screen for the entry of the handwritten symbol; 

using a hand-held means to write or draw the handwritten symbol in the window area; 
55 digitising signals representative of the handwritten symbol; and 

entering the digitised signals into the computer system. 
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2. A method according to claim 1, further including displaying the image of the handwritten symbol on the 
screen as it is written. 

3. A method according to claim 1 or 2, in which at least two handwritten symbols are entered in the 
window area. 

4. A method according to any of claims 1 to 3, in which the handwritten symbol comprises a number, a 
letter, a Kanji symbol, a user-defined symbol, a command or editing symbol, a graphic image or a 
drawing or the like. 

5. A method according to any of claims 1 to 4, in which the handwritten symbol is identified as 
corresponding to a font symbol, and the corresponding font symbol is entered into the computer 
system. 

6. A method according to any of claims 1 to 5, in which the predetermined command in response to 
which the window area is created is input as a handwritten symbol by a user using the hand-held 
means. 

7. A handwritten symbol recognition apparatus comprising: 

a display screen (18,20); 

a hand-held means (16) for use by a user to write or draw a handwritten symbol on or over at least 
a portion of the display screen; 

a digitiser (18,64) for digitising the handwritten symbol; 

a processor (14) coupled to the digitiser for determining the identity of the handwritten symbol; and 
a means coupled to the display screen for creating, in response to a predetermined command, a 

window area in at least a portion of a display area of the display screen for the entry of a handwritten 

symbol for recognition or otherwise. 

8. An apparatus according to claim 7, comprising a display means (14) coupled to the digitiser and to the 
display screen for displaying on the display screen an image of the handwritten symbol as it is written. 

9. An apparatus according to claim 7 or 8, in which the window area on the display screen is usable as 
both: 

(i) a window area for entering a handwritten symbol during a first time period following execution of 
the predetermined command; and 

(ii) a display area for displaying predetermined text during another time period. 

10. An apparatus according to any of claims 7 to 9, in which the window area is free of text and other 
information. 

11. An apparatus according to any of claims 7 to 10, in which the window area can be of various sizes and 
shapes. 

12. An apparatus according to any of claims 7 to 11, in which the handwritten symbol comprises a number, 
a letter, a Kanji symbol, a user-defined symbol, a command or editing symbol, a graphic image or a 
drawing or the like. 

13. An apparatus according to any of claims 7 to 12, in which a document for editing by insertion of a 
symbol is displayed on the display screen, the window area is created near to the portion of the 
displayed document where the symbol is to be inserted, and the symbol is entered by being written or 
drawn in the window by a user using the hand-held means. 

14. An apparatus according to any of claims 7 to 13, in which at least two handwritten symbols are entered 
in the window area. 

15. An apparatus according to any of claims 7 to 14, in which a soil key may be configured as an area 
within a display area of the display screen, wherein when a user selects any point within the soil key 
area by means of the hand-held means, a predetermined command is executed. 
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16. An apparatus according to any of claims 7 to 15, in which the predetermined command in response to 
which a window area is created is entered by a user as a handwritten symbol. 
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I - Printed Circuit Board 

The Hoard's size, cost, power consunption, ind 
reliability are critical to the sucess of our product. 

The board mis t fit in our package, which IIhHs Us 

diwensions to southing lilte IB 1 5 , or about 50 square 
inches. Room mist be available for dealer installable 
options such as additional Hewory and Hodew - possibily as 
piggy back boards.^ 

Power consuHpt ion Hust be low enough so that the 
Lat tries in the product can provide for about 8 hours of 
operation and about 2 weeks of data retention. These are 
exclusive, i.e. 8 hours of operation followed by 2 weeks of 
data retention is required. Uith this kind of power 
consuHption heat should not be a problem. 

Given our cost and power requircMents we expect that we 
will be using CHOS technology through out. We also expect 
to use Surface Mounted Techno logy(SKT). SHT allows a denser 
packing, wore shock resistence, and lower Manufacturing 
cgslt, although Initial costs iuvj Le sli'/htly higher. The 
final decision on Shi will depend upon the availability of 
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reliability are critical to the sucess of our product, 

The board must fit in our package, which limits its 
dimensions to something like Jfr 1 5\ or about 50 sguare 
inches. Room must be available for dealer installable 
options such as additional Hewory and modem - possibiiy as 
piggy back board. Jlaws-Uii-iiiiP 

PcH?r consumption must be low enough so that th? 
battries in the product can provide for about 8 hours of 
operation and about 2 weeks of data retention. These are 
exclusive, i.e. 8 hours of operation followed by 2 weeis of 
data retention is required. Hith this kind of pomer 
consumption heat should not be a problem, 

Given our cost and power requirements we expect that we 
mill te using CiiW technology through out. He also expect 
to use Surface Mounted Technology^!), SHI alloHs a denser 
packing, were shock resistence, and lower manufacturing 
costs, altlijugh initial costs mau b; slightly higher. The 
final decisien on SKI Hill depend upon th? availability of 



43 



/4, 



EP 0 645 730 A1 
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